Text-to-3D AI is getting better by the day, or so it feels. While 2D generative AI such as Midjourney and Stable Diffusion have grabbed the headlines, 3D modelling has gone under the radar. But now researchers Junzhe Zhu and Peiye Zhuang have created HiFA, and it's an impressive 3D AI generator, even at this early stage.
HiFA can create high-definition 3D models for use in video games, 3D scenes and animation. Some may debate what games can use these assets, but this AI is offering good results. The controversy around using AI is still ongoing, with artist Greg Rutkowski recently telling me it was "terrifying" to have his work used in an AI without permission. But the technology is advancing fast.
The difference between HiFA and its rivals comes from its use and optimisation of pre-trained text-to-image generative AI, such as Stable Diffusion. While some text-to-3D AI can create models they are built on 2D-trained renders and so can have inconsistencies. Below HiFA outlines its approach and why its 3D AI models look better than most.
"To address these limitations, we propose a reformulation of the optimization loss using the diffusion prior. Furthermore, we introduce a novel training approach that unlocks the potential of the diffusion prior. To improve 3D geometry representation, we apply auxiliary depth supervision for NeRF-rendered images and regularize the density field of NeRFs. Extensive experiments demonstrate the superiority of our method over prior works, resulting in advanced photo-realism and improved multi-view consistency."
I'm personally impressed, the 3D models being generated by this AI look good. If you don't want to use the final model you can simply take and adapt the 3D mesh and polish it or create something new from it.
Our recent 'How to use Adobe Firefly tutorial' revealed our artist Ken Coleman uses AI to generate elements for concept art and mixes in ZBrush and Procreate into his workflow, I could see text-to-3D AI generated models working well with this workflow. This AI from Adobe is seen as the more ethical model, if you want to learn more read our feature 'Adobe Firefly explained'.
If you want to dig deeper into generative AI then read our guides to the best AI generators and discover how AI generators compare, too. But none of these really deliver text-to-3D models like this new AI generator proposed by HiFA, you can see more models created in this AI app below, or discover how it all works at the HiFA website.
