Think Midjourney Meets The Metaverse

Sooner than you might have expected and probably quicker than you were planning, NVIDIA just announced GET3D an AI model for generating virtual world objects — think Midjourney meets the Metaverse:

is looking to take the sting out of creating virtual 3D worlds with a new model. GET3D can generate characters, buildings, vehicles and other types of 3D objects, NVIDIA says. The model should be able to whip up shapes quickly too. The company notes that GET3D can generate around 20 objects per second using a single GPU… The model can create objects with “high-fidelity textures and complex geometric details,” NVIDIA’s Isha Salian . The shapes GET3D makes “are in the form of a triangle mesh, like a papier-mâché model, covered with a textured material,” Salian added.

The NVIDIA blog post doesn’t mention anything about rigging these 3D models to character/avatar skeletons, so that might not be part of the AI’s purview. In any case, this is the kind of thing that metaverse artist and technologist Jeff “AM Radio” Berg was anticipating only two months ago:

It is now possible, using programs like Craiyon, a text-to-image generation tool powered by AI (a variation of OpenAI’s DALL-E), to automatically create images strikingly similar to the Second Life installations which took him many hours if not days to create in 3D. “I wondered the impacts it could have on metaverse space creation,” as he mused to me recently, sharing the results above. “3D geometry rather than a generated image from plain text input doesn’t seem like a huge stretch.”

It’s not a stretch at all, as it turns out. But back in July, at least, AM Radio saw potentials, not threats to 3D creators like him:

As we try to even define the it, the indescribable as if we’d discovered and trying to explain a new color, a new way of being. That gaze includes that scary and gripping view into the uncanny valley of machine learning. People talk about the death of creativity and art in light of machine learning, and yet I sit in awe at the work ahead for every artist to explore and uncover the virtual as it begins to gaze back at us.

… Machine learning fills us with fear, it makes us turn away, we dismiss it. That is exactly the marker for an artist to plant a flag and say it’s time to turn towards this and find out how very superb but terrifying this fine tiger really is.