Meta Reveals ‘WorldGen’ Tool to Generate VR Worlds from AI Prompts

By
Scott Hayden
-
18

Meta announced WorldGen, a new AI tool that could soon let you generate navigable 3D worlds in minutes from a single text prompt.

The News

Meta Reality Labs announced WorldGen in a blog post, a research-stage system for generating fully navigable, stylistically coherent 3D worlds from a single text prompt.

As outlined in WorldGen’s research paper, instead of producing only a single viewpoint or small environment, the AI-driven system creates large, consistent scenes up to 50 × 50 meters that you can walk through, explore, and load directly into engines like Unity or Unreal Engine.

The pipeline combines several components: procedural layout generation, image-based planning, diffusion-driven 3D reconstruction, navmesh extraction, scene decomposition, mesh refinement, and texturing, Meta says.

Additionally, Meta says WorldGen can also decompose scenes into objects using an accelerated AutoPartGen process, making the environments more reusable and editable.

It’s not out yet though, as Meta says there are several limitations to wide-spread usage:

“Currently, WorldGen relies on generating a single reference view of the scene, which restricts the scale of scenes that can be produced,” Reality Labs’ paper says. “Large open worlds spanning kilometers are not supported natively and would require generating and stitching multiple local regions, which risks introducing non-smooth transitions or visual artifacts at region boundaries.”

Other limitations include the inability to model multi-layered environments, like multi-floor dungeons or seamless interior-exterior transitions. And unlike human-created environments, WorldGen doesn’t reuse textures or geometry, which is often done for rendering efficiency—something Meta says they’re exploring in future versions to push scalability.

SEE ALSO
Meta Reportedly Reshapes Metaverse Division Amid Leadership Shifts, Putting AI at Its Core

My Take

Horizon Worlds needs high-quality content to attract and keep users coming back for more. Although I can’t say rando-generated AI worlds will solve that problem, doing some of the grunt work of creating set pieces could demonstrably improve the baseline of current worlds.

To boot, the company has already released a host of AI features for Horizon Worlds creators, including an AI-powered ‘Creator Assistant’ co-pilot, as well as AI-driven 3D mesh and texture generation, typescript code creation, sky and audio generation for sound effects and ambient environments; WorldGen ostensibly packages a lot of these disparate systems into a monolithic prompt box.

Granted, it’s not out yet, although these early steps do feel a bit like inching up to the cusp of fully AI-generated games, and not just 50 × 50 meter levels. Whatever the case, it appears Horizon Worlds hopes to one day play host to worlds essentially vibe developed into existence.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.

Scott Hayden
Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Stephen Bard

    We are still waiting for the Horizon Desktop Editor to be replaced by the Horizon Studio, so that you can AI generate incredibly detailed unreal Horizon Worlds spaces and objects instead of just Islands. Maybe Worldgen will be a part of the Horizon Studio. Meanwhile I discovered an app that generates fairly detailed walkable VR spaces called Marbles at the "WorldLabs" website. You go to the Marbles page and click on the headset icon on the many sample images and suddenly you are transported inside a VR space where you can look/move all around. Then make your own worlds with a prompt or image. The spaces have some splat artifacts, but are still amazing. Meta should buy this tech and host it as another Horizon Worlds category.

  • Christian Schildwaechter

    TL;DR: this adds to existing tools for world generation, but lacks the performance optimizations absolutely required for mobile VR.

    Technically this is very impressive, but I have doubts how much this will actually help. It somewhat "solves" the modeling and level design part, but that has been already covered by purchasable assets and level editors with rule based environment generation. You can get tools for drawing roads and rivers, and have vegetation spread over the world automatically, with some flower growing next to water, or trees grouping together. Creating a level with navmeshes telling NPCs where to go in for example Unity is still some work, and you'll have to find matching assets, but there are tons of these, often already optimized for mobile platforms. And that is the second big issue:

    And unlike human-created environments, WorldGen doesn’t reuse textures or geometry, which is often done for rendering efficiency—something Meta says they’re exploring in future versions to push scalability.

    If you buy a properly optimized asset pack, it will come with for example houses created from smaller modules that are reused over and over to reduce render time, as GPUs can redraw the same object rather fast, compared to drawing a second one, even if it is basically identical. And usually all the textures will be placed in a few atlases to reduce the amount of materials, as large amounts of materials require more rendering passes, which are prohibitively expensive on mobile.

    So what I'd expect is a flood of small walk-through demos with nothing to do in, like in the very early consumer VR days, where a lot of people just took some demo scene from an asset bundle, placed a character controller with a VR camera head model in it, and released the whole thing as a free app. This was interesting for a while, because you could walk through a decent looking virtual world, which was still a new experience, but that was pretty much it. There was nothing to do, nothing to interact with, and the step from this to something that would qualify as a game or valuable experience required a lot of extra effort.

    No doubt AI generation can speed up time consuming processes, but there are already a lot of tools that help with that in a way that is much easier to control than having to tweak AI prompts until the result matches what you were looking for. VR is a lot about interaction, which is what made HL:A a great experience. Just generating and populating an open world or a dungeon with random assets doesn't create an interesting experience, and we already have tons of specialized tools for that which actually create performance optimized results that might work on a mobile headset.

    So if Meta is hoping to use these AI tools to compensate for a current lack of content in Horizon Worlds, they'll have to add a lot more before a regular user will be able to create something interesting with them. Otherwise all we'll get will be tons of visually appealing scenes to walk around in, either very small or with rather bad performance, but overall shallow experiences. Basically VR AI slop.

  • Oxi

    I check out of low quality VR content incredibly fast so the idea of just crowding up VR platforms with slop sounds absolutely terrible.

  • rabs

    They are trying to bury themselves under shovelware and slop content, because the problem surely is that there aren't enough already.

    • JoeD

      Wow. Such edgy commentary.

      • Gato Satanista

        Wow. AI bro detected

  • xyzs

    What about releasing a good quality OS, not bloating/forcing meta products people do not want, with a well-made UI, with a store where only quality game are allowed ?
    That could be a nice focus, no ?

    • Rayza

      MetaOS is a janky piece of crap, i can't wait to switch to a better headset. Hopefully GalaxyXR is the one

  • Nothing to see here

    Meta is definitely going in the right direction for content creation. I sense that there will be a break through at some point and it will allow for fully detailed models to be generated quickly. That will likely signal the end of billion dollar games taking hundreds of people many years to complete and open up the possibility of games being generated on the fly based on the player's actions.

    • Rayza

      And then you returned to the real world

      • JoeD

        What's funny is you think your comment is smart. It's not. You are generally an unintelligent person. Rayza is 100% correct. AI is a tool, much like you. But AI, unlike you, is actually a productivity increaser. Just like any tool it can be used to generate garbage. Anyone with a computer could create crap long before AI came on the scene. And people just like you made uneducated comments like this when computer graphics were in its infancy, especially 3D graphics. AI is democratizing computer programming, in essence. Before AI, you needed to speak in code to communicate with the computer, or use an interface designed by coders (Photoshop/blender/Excel). The coders used programming language to convert their thoughts into machine language, which is what computers understand. NOW, anyone can simply use basic English (or whatever) to communicate with a computer to have it do what it wants. This will also democratize nearly everything. Guy wants to make a game but has no programming experience or money to hire a coder? No problem, he takes his idea, his mechanics for the game, and tells the AI what to do. Of course there will be crap that people mKe,bthats the way things always go until the market sorts it out. But you people, no different than the luddites of old, would be happy if we were all living in mud huts just so that no one "lost" a job due to technology. You nerds are all into Marvel. How do you think Tony Stark was able to create the things he did? Because he had an an AI driven, automated design and machine shop in his basement. This is the reality. That ONE guy, able to create what he did due to the ability of AI to exponentially increase human productivity. Luckily, people like you, and your uneducated opinions, like those before you, will be left in the dustbin of history.

  • I don't agree: the more AI tools Horizon Worlds is adding, the more we are seeing the publication of terrible AI slop worlds. These worlds add server costs to Meta, make discovering good worlds difficult, and just provide a bad user experience to whoever wants to use Horizon Worlds

    • JoeD

      Yeah, cause user generated content without AI is so good. You people truly are nuts. AI is in its infancy and can already outperform the vast majority of Peopke as far as content creation. You people are no different than the people decrying computer graphics when they first came on the e scene, and 3D in particular.

      • Gato Satanista

        The only people that AI can already outperform is you.
        AI lacks real creativity and it needs humans to create anything worth

        So delusional. Go lick Altmans and Zucks boots bro

  • david vincent

    Yet another kind of AI slop, great

    • JoeD

      Do you guys meet up in someone's mom's basement and decide what kind of st00pid phrases you're going to use when new tech comes out that threatens your existence in no way whatsoever?

      • Gato Satanista

        We got an AI bro here, guys. People who lack imagination and intelligence get impressed by mediocre AI slop way too easily.

  • Rayza

    Looks embarrassingly bad.