Can Midjourney Actually Build a Holodeck Despite Technical & IP Infringement Hurdles? Reid Southen Has Doubts.

Matrix Resurrections artwork by Reid Southern (copyright Warner Brothers)

Midjourney plans to make some kind of open world "holodeck", as CEO David Holz confirmed with me earlier this month. But that actually just begs the question: Can it even feasibly do so? 

The company is already careening toward IP infringement challenges for its 2D image creator, based on reports of rampant copyright infringement, including a lawsuit led by artist Jon Lamm, alleging that Midjourney staff (including Holz) have intentionally "laundered" the work of thousands of artists into its training database without consent.

So what happens when Midjourney tries creating a platform with immersive 3D content?

Film industry concept artist David Southen, who's worked on Marvel and DC movies, not to mention the Matrix and Alien franchise, is highly doubtful.

"In terms of them pulling something like this off without IP issues?" as he puts it. "They can't even pull off what they're currently doing without IP issues, and I doubt they're being more careful with their dataset than previously, so I would say no."

Southen would know, and not just because of his unimpeachable geek artist cred. He recently wrote this report with renowned AI ethicist Gary Marcus, highlighting the existing problem:

These results provide powerful evidence that Midjourney has trained on copyrighted materials, and establish that at least some generative AI systems may produce plagiaristic outputs, even when not directly asked to do so, potentially exposing users to copyright infringement claims. Recent journalism supports the same conclusion; for example, a lawsuit has introduced a spreadsheet attributed to Midjourney containing a list of more than 4,700 artists whose work is thought to have been used in training, quite possibly without consent. For further discussion of generative AI data scraping, see Create Don’t Scrape.

How much of Midjourney’s source materials are copyrighted materials that are being used without license? We do not know for sure. 

More on this on Marcus' blog.

And unlike most 2D artists, high quality 3D files tend to be owned by folks like major game publishers and movie effects houses who won't exactly be thrilled about giving away their content for free — and have full-time lawyers on staff to protect it.

As for the technical challenges, Reid continued:

Midjourney developers caught discussing laundering, and creating a database of Artists (who have been dehumanized to styles) to train Midjourney off of. This has been submitted into evidence for the lawsuit. Prompt engineers, your “skills” are not yourshttps://t.co/wAhsNjt5Kz pic.twitter.com/EBvySMQC0P

— Jon Lam #CreateDontScrape (@JonLamArt) December 31, 2023

"I think David Holz is talking a much larger game than he's capable of backing up. He's been shown in the past seemingly brainstorming and spitballing ideas openly on Midjourney's office hours, but nothing about the technology they've demonstrated would indicate we're anywhere close to 'holodeck' technology. Much larger corporations with far more employees and far deeper pockets have been working towards similar goals for much longer, and haven't gotten anywhere close to a 'holodeck'.

"I don't doubt that David has some sort of 'realtime' version of Midjourney's image generation in the pipeline, whether it's something akin to video game rendering, or something with VR, but calling it a 'holodeck', which is a very specific type of technology from Star Trek, is likely very misleading."

The other issue is computing cost required by generative AI.

"I don't know if Midjourney has proven to be profitable yet," as Reid puts it, "but AI companies like Midjourney need to keep raising capital in order to cover their R&D and immense server costs, and I personally think that David has a vested interest in keeping the hype train trolling at all costs."

I asked David Holz for a comment to all this yesterday and will update this post if I get a reply.

For my own part, all this talk of a holodeck misses the fact that there are already thousands if not millions of people creating relatively high quality immersive content on metaverse platforms… and actually enjoy doing it, and make money from doing so. So a holodeck sure seems like a legally and technically expensive problem where a democratic solution already exists. 

As for Reid, follow him on Twitter/X below!

So not only do they need copyrighted material for their technology to function, but energy demands are getting so high (think small country), they also need a nuclear fusion breakthrough to sustain it. The whole thing is such a joke. https://t.co/VczeiPGj34 pic.twitter.com/yzeZyiwCED

— Reid Southen (@Rahll) January 22, 2024

Read More: nwn.blogs.com