ChatGPT is Currently Mangling Metaverse History. But Can It Be Fixed?

Beverly Millson, an emerging tech consultant, was recently playing around with ChatGPT and she decided to ask the AI about her Second Life avatar, Bettina Tizzy. Through that avatar name, she edited Not Possible in Real Life, which up until 2009 was a must-read site covering the works of an emerging new art form: Immersive expression created in metaverse platforms like Second Life.

But in reply, ChatGPT gave Beverly’s avatar a small if not exactly accurate promotion — the creator of the very term, metaverse: 

As I’ve mentioned before, asking ChatGPT about a topic where you have some deep expertise exposes just how much the program is mostly a glorified, easier-to-read version of Google search — and equally prone to coughing up false-positives or extremely mediocre results. (ChatGPT’s definition of the Metaverse is not very accurate to the source material either, but leave that to one side.)

I also know enough to infer how ChatGPT probably went so wrong, in Bettina’s case — ChatGPT has likely been trained on blog posts like this one where she’s been described as a “metaverse art maven”, and it made the next illogical leap.

Amused, Beverly recently sent me this result for a chuckle. But it also makes me somewhat concerned how large language models like ChatGPT are training on important but somewhat obscure subjects: In this case, metaverse history.

For instance (also via Beverly), here’s how ChatGPT describes the SL artist known as “AM Radio”:

Contrary to what ChatGPT outputs, AM Radio is mainly known for creating rural virtual scenes — which were even profiled in the New York Times — and his real identity has been reported: I wrote the profile on Polygon, another large news outlet, which did so.

Here’s my concern: By now it’s pretty well known that ChatGPT will often “hallucinate” bad data, and so needs its output fact-checked. But because virtual world history and culture is a second (or third) layer of knowledge beneath real life, so to speak, I strongly suspect LLMs are not sophisticated enough to train on it.

I’m not sure if there’s a good immediate solution to this problem, so I’d just be extra cautious when using ChatGPT in a metaverse context for anything more than some goofy fun. At least ChatGPT got metaverse artist Bryn Oh’s bio right (above). Likely because Bryn is still highly active online.   

Googling “who is the real person behind the SL avatar am radio”, by the way, gave me the right answer in under two seconds.