AI in 3D Modeling

Reflective Essay

By Andrew Truax

Introduction

AI in creative production is being increasingly used to transform 2D images into 3D models, which will reconstruct depth and dimensions using the flat visuals of an image. This technology enables artists and designers to quickly generate realistic 3D representations from simple sketches or photographs. I will go over my central and several sub questions about AI and the implications on jobs for 3D modeling.

The importance of this exploration is to test these AI models and see how well the AI of today can re-create the 3D models using only 2D images using several similar applications.

Reflective Analysis

The central question to all of this is if AI can use 2D images to create 3D models, will this practice take away jobs from 3D modelers? This question raises several more sub questions such as:

  • How does the quality of 3D models created by AI compare to those created by human modelers?
  • What specific tasks do 3D modelers perform that AI currently cannot?
  • What are the current limitations of AI in creating accurate and detailed 3D models from 2D images?
  • How could AI potentially enhance the creativity and efficiency of 3D modelers rather than replace them?

    • To make this all happen, I ended up using several free images to model AI converters. This includes Meshy, ZoeDepth and DreamGaussian. I also used Chat GPT to enhance and extend my thinking as well as improve my website code.

    The integration of AI in creating 3D models from 2D images represents a significant advancement in digital art and design technology. This process typically involves using machine learning algorithms to interpret and extrapolate the depth and structure from flat images, effectively rendering them into three-dimensional models. This technology could apply to many fields such as reality, video game design, film, and even in medical visualization, where it enhances both creativity and functionality.

    There are a few ethical concerns regarding AI in this corner of the industry that is 3D modeling. Firstly, there's the issue of authorship. AI can replicate and manipulate existing pictures without clear consent from the original creators, potentially leading to intellectual property disputes. I don’t think these AI models use anybody’s work so it could not break copyright. However, if someone takes and uses an image a 3D modeler created on the internet and puts it into an image to model converter, then that would arguably be stealing in my point of view. There's also the concern about the authenticity and originality of art, as AI may dilute the unique touch of human artists by automating creative processes.

    Furthermore, the accessibility of such technology can both democratize art creation, making sophisticated tools available to a wider audience. Ensuring transparency in how these AI systems operate and establishing robust ethical guidelines are crucial steps in addressing these challenges and fostering a respectful integration of AI in artistic practices.

    Creative Project Execution

    To get the results I desire, I must find an AI program that can do what I'm looking for, which is converting 2D images into 3D models. I found Meshy and this was the only one out of my first batch of known programs that made it convenient to begin generating.

    There were other programs such as Kaedim and Alpha3D but those either did not work or were not convenient. Kaedim required me to pay $300 as an Indie each month to be able to use their converter. Alpha3D, however, could not log me into the account I had just created and when I did finally get it working, I learned that they were only capable of converting images of sneakers and sofas into 3D objects. Aside from that, they were a text-to-image converter program.

    While I was using Meshy, I was asked to find an image that was just a single object in one image, standard front view, and no background so that’s what I created for this experiment. I rendered out an image of this model called Sadie with the camera in orthographic mode. This helps remove the 3rd dimension and makes her 2D. After that, I enabled the transparent background.

    The results from Meshy were disastrous. The entire model looks stretched to be a little more squared than it is supposed to be round. Sadie is missing a few fingers too. The colors look off. You can tell what part the generator took from the render and what was made up using what’s on the render. The face is completely flat as well. This AI cannot tell the distance between different aspects of the image even when the model in the image has lighting. It is mainly making guesses about what the shape of the subject is and getting it very loosely right.

    Meshy wasn’t enough. I needed to find more AI converters. It took a while to find what I was looking for, but I eventually found 2 more, ZoeDepth and DreamGaussian. I found them both coincidentally on the same website called “Hugging Face.” When I tested ZoeDepth, the result didn’t work very well. There was a black background that was a part of her model and it just felt very low quality. DreamGaussian, on the other hand, wasn’t too bad. It is on a similar level of quality to what Meshy offered. The model is a bit more accurate but the texture quality is easily the worst. None of these options are good but most of them weren’t genuinely bad like ZoeDepth was.

    Conclusion

    The exploration of artificial intelligence's impact on artistic production, specifically in transforming 2D images into 3D models, highlights both the capabilities and limitations of current AI technology. AI applications like Meshy, ZoeDepth, and DreamGaussian, while they vary in effectiveness, they underscore the technology's potential to significantly alter the landscape of 3D modeling and design. These tools demonstrate AI's ability to expedite the creation of 3D models from simple sketches or photographs, which can enhance creative workflows and open new possibilities for artists and designers. However, the quality of these AI-generated models often does not match that produced by skilled modelers who are real people, which indicates a gap that AI has yet to bridge.

    I don’t believe AI will take jobs away. Not exactly, anyway. I think it will be very difficult for these AI models to generate 3D models in the meantime, but they will catch up just like Sora did. I have heard and seen some AI models being able to replicate an image’s likeness as a model very well just by requiring multiple photos of a subject instead of one image. I think one day, more of these AI models will catch up and let you submit more than one image at a time for more accurate results. If 3D artists want to continue to survive in this capitalist society, they don’t have a choice but to learn this type of AI if they want to remain relevant in this industry.

    I learned that AI could turn 2D images into 3D models, but I also realized that the technology is very far away from being able to be used in the industry. unless there is already a better AI model being used now that I am not aware of. AI can be used for good and to be able to assist us in getting started on a project, but I know they should prepare to use this in general or else many talented 3D modelers will get left behind.