WATCH ChatGPT Generate Working Terrain in a Virtual World When Prompted

When I recently wrote about all the impressive innovation happening on Wolf Grid, an OpenSim-based virtual world developed by small team, I didn't even mention one of its coolest breakthroughs:

Wolf Grid has an option which uses ChatGPT to generate usable terrain with a prompt.

Watch above, with the money shot happening around 6:30 in. In the first demo, requesting a "cat" turns the land into a mountain range that's a picture of a cat (kinda sorta), while requesting a "maze" actually generates a working maze that springs up on the land.

"Basically," lead developer Lone Wolf tells me, "we have some software between the AI and the grid that deals with any issues."

From the user perspective, they communicate with Bobby, the Wolf Grid's AI assistant, which is integrated with ChatGPT.

"[We] use our software to decide what to do? Do we need to generate an image? Do we need a terrain? Do we need to answer a query? Then it works out which 'bit' of ChatGPT to talk with, meanwhile recording information so it's able to know who said what to him, and then interpreting the information back from the AI and translating to a useable thing."

Wolf tells me Bobby will eventually be able to make all this work on the grid live. Unless I'm mistaken, this is the first instance of a virtual world actually using ChatGPT to generate physics-enabled terrain, as opposed to non-interactive, diorama-type backgrounds. (Correct me if I'm wrong, readers!)

Getting generative AI to create working virtual world terrain is more challenging than it might seem, but Mr. Wolf tells me they've done that:

"Bobby's like, 'here's a cat' and we go, 'OK thanks Bobby, but we need to just change some of it to make it work as a terrain, even then more detailed images can look a bit weird as a terrain… So we 'help' the AI get the image right. But the user doesn't see the conversation."

So yes, users can actually go in and navigate the maze that Bobby/ChatGPT generated. "As I said in my video you might want to put a prim [path[ down if people were to walk through it." Or, adds Wolf, you could flood the bottom with water and have people navigate it in small boats.

Pretty exciting use of ChatGPT. And unlike some other ChatGPT applications, it decidedly doesn't disrespect Scarlett Johansson.

Please support posts like these by buying Making a Metaverse That Matters!