‘Horizon Worlds’ is Getting AI-powered NPCs Soon That Talk, Guide and React in Real Time

22

Meta laid out the next step of its AI integration strategy for social VR platform Horizon Worlds, noting that “very soon” creators will be able to add AI-powered NPCs to their worlds.

Earlier this year Meta introduced non-embodied NPCs, essentially letting Horizon Worlds creators plug in ‘background’ AI agents, which were meant to offer help, but not exist as avatars.

Now, Meta announced in a developer blog post it’s soon rolling out “fully-embodied conversational [large language model] NPCs”, which means makers will soon be able to populate their Horizon Worlds creations with NPCs ostensibly sourcing responses from Meta’s latest Llama LLM.

Meta says this will allow NPCs to hold dynamic, unscripted conversations with players, mix scripted dialogue with LLM-generated replies, and use different AI-generated voices from a built-in library. Check out the character builder in action below:

 

Creators will also be able to define NPCs by name, and include things like backstories, personality traits, and specific dialogue styles, which Meta notes could serve as quest givers, guides, lore-deepening characters, shopkeepers, or bosses that react dynamically.

Meta isn’t stopping there either. Later this year, the company says they’re also adding the ability for NPCs to trigger in-world actions, dynamically converse with real players, and “more.” We’re sure to learn more at the company’s annual developer conference, Meta Connect, when it kicks off September 17th.

SEE ALSO
'Hello Kitty' is Getting Its Own Social VR Game This Year in 'Skyland'

This comes amid continued efforts to boost engagement in Horizon Worlds, and further differentiate it from other platforms too—like Rec Room, Roblox, and VRChat—all of which feature scripted NPCs.

To boot, earlier this month Meta released two major generative AI features (Creator Assistant and Style Reference) for its Horizon Worlds desktop editor, aimed at streamlining the development of user-generated environments.

In all, Meta is striving to arm Horizon Worlds creators with better desktop maker tools that require less technical knowledge, effectively letting them build more visually-rich (and soon) more narratively-rich games and experiences.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.

Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Christian Schildwaechter

    So instead of mostly avatars that turn out to be annoying kids, it will now be populated by both avatars that turn out to be annoying kids and avatars that turn out to be hallucinating AI chatbots. I'm sure this is what was missing to finally turn Horizon Worlds into a popular social VR space.

    • lol exactly

    • VRDeveloper

      But you have to understand one thing: these big companies think in absolute numbers. So, if you’re an entrepreneur, the idea of increasing production even with a low-quality product, meaning massive investment in AI, seems very obvious.

      So we need to understand why these large companies make this kind of mistake. From our perspective, as people who actually spend money and engage in good experiences, watching this feels like watching a company burn cash. But I think it’s understandable, because for the average entrepreneur, it feels like they’re doing everything right. That’s why it’s possible to understand why companies fall into these mistakes.

      Still, it’s sad to see that the VR landscape has become filled with low-quality content, with the very companies themselves flooding the platform with junk. I just hope this doesn’t end up destroying VR again.

      • Christian Schildwaechter

        TL;DR: It's not a mistake, it is very deliberate for actually good reasons. It just sucks for VR users.

        After the 2021 Metaverse hype incl. the company rebranding faded without leaving a mark, Meta switched their story to AI everywhere, which pleases investors, and at least with the Meta Ray-Bans also works for XR. But the goal is still the metaverse, or more precisely the financial benefits from providing core services to the Metaverse.

        This requires large user numbers that VR didn't manage to provide. So Meta is enviously looking at Epic's Fortnite that launched as a stylized Battle Royale game in 2017, but since then expanded into something much closer to the metaverse than Meta ever achieved, with lots of other game modes and events. 12.3M players came to Travis Scott's "Astronomical" concert in Fortnite, more than the entire active Quest user base.

        The free-to-play game/world is also a money printing machine, generating around USD 5B in revenue each year, while Meta Reality Labs loses twice that each year. Fortnite makes so much money that it made sense for Epic to get into a years long legal and political lobbying fight with Google and Apple to no longer have to pay them 30% of their income. And Epic effectively won, with Google in the US being forced to allow 3rd party app stores, incl. the one from Epic, and the EU now pushing legislation to force competing app stores on all platforms, with the US possibly following.

        So from Meta's long term strategic perspective, throwing everything at Horizon World and pushing esp. the mobile phone and web version absolutely makes sense. They desperately need more users/engagement and content creators, so populating it with AI NPC makes sort of sense to give people something other than screaming kids to interact with, while also leveraging Meta's huge investments in AI.

        This is only a mistake if what you are looking for is high quality VR experiences, but that was never what Meta was aiming for. VR was just a way to get to their actual goals that didn't work out, so now they are trying something different to attract the Fortnite masses, which just happen to have very different spending habits and overall priorities than VR enthusiasts.

        • VRDeveloper

          Christian, I completely agree with everything you said, and as always your comments are very sensible. You are clearly a professional in the field as well.

          That said, when I mention a mistake, I mean Fortnite. For nearly a decade the industry has been chasing the “perfect live service” model, convinced it guarantees infinite revenue. From a business perspective it makes sense: AI, increased production, casual user engagement, investment in live service. They look at charts and think, “Roblox and Fortnite print money, so we should do the same.”

          But here’s why it’s a mistake: in the last 10 years, most studios that switched to live service have failed. Money alone is not enough. You need a product with quality and soul, backed by a passionate, talented team. That’s what made Fortnite succeed, and why so many others failed. Meta is following the same empty path. They turned the Quest into a soulless product, and that’s the critical point.

          How can creators like me trust a platform that only pushes free live service games, while ignoring players who want meaningful experiences? We won’t. More technical developers are leaving for Steam, which is already proving to be the stronger platform for VR, as shown by Into the Radius 2 — a purely technical game that is thriving on PCVR.
          This trend will kill mobile VR, if it hasn’t already. Who will spend $400 on a headset filled with AI-generated garbage? I open my headset and see nothing but trash, and I even question if it’s worth having my serious title sitting next to it, as it damages its image. To me, this is a brutal long-term mistake.

          They won’t be Fortnite. They should be investing in indie studios like Tactical Assault VR, developers who actually understand how to make good games. Filling the platform with quality experiences is what would secure a solid future for VR.

          • Christian Schildwaechter

            TL;DR: Neither Roblox's nor Fortnite's success are the result of "quality and soul", they are mostly accidental successes with sound business decisions following. Which is why we see so many seemingly doomed "me too" projects. You only need to win once, as long as you can grab a huge user base. Large scale gaming is driven by Excel sheets, even if gamers want to believe in the passionate small developers winning in the end.

            I wouldn't exactly describe what drives Roblox success as "quality and soul". And Fortnite actually started in 2011 as a mix of Minecraft and Left 4 Dead, but six years into its development, PlayerUnknown's Battlegrounds became a sudden success, selling 5M copies in just a few months. Epic basically instantly threw out their original concept, pretty much ripped off the whole Battle Royal concept from PUBG, placing it into a more cartoony world, and in a last minute decision made it free-to-play.

            Implementing the Battle Royal mode only took about two months. You can call that a "passionate, talented team", but "ruthless opportunistic corporate behavior" is closer to the truth. People actually liked the game, esp. not having to pay for it, as at that time PUBG cost USD 30 on PC/Xbox, and Fortnite gained 10M players in just two weeks. PUBG finally became free-to-play in 2022, but by then Fortnite pretty much owned that market on all platforms.

            The idea that good content, loving your product and listening to the users will ultimately win is a nice fairy tales gamers want to believe, but it is mostly a fairytale. For every single dev Undertale, or Hello Games that kept working on No Man's Sky for a decade and turned a turd into gold, there are one hundred low effort free-to-play Android titles pushed into the marked with huge advertising budgets, making more money.

            And yes, all the big studios have been trying to come up with the next live-service games raking in the big bucks, and most utterly failed within a very short time. But the brutal truth is that those few that succeed make so much more money, that studios that actually do the math will still try. It's basically venture capitalist game development, nine out of ten investments are duds, but the 10th pays for all the rest.

            Meta will most certainly try for that, and if Horizon Worlds fails, they will try again, because they aren't just looking for one live-service game, they want to become one of the main live-service platforms. Which will be worth hundreds of billions in development costs if they succeed. It is of course a giant gamble, with no guarantee that they will win, but they knew this from the beginning. And they currently have a money printing machine with Facebook and Instagram ads, and will use that money to make huge bets to ensure they get another money printing machine in the future.

            I'm sure there are a lot of people at MRL that are very passionate about VR and high quality VR games, and that they'd really love to turn Quest into a great experience for VR enthusiasts. But too few people care about VR despite years of massive improvements. And it most likely boils down to people not wanting to strap things to their faces, so this is a fundamental problem that won't be resolved.

            This makes VR a niche that simply cannot deliver anywhere close to the user numbers that other (mobile) live-service games require to work, so it will not see more huge content investments. And even within the VR niche, the must successful games aren't the passion projects of indie developers that love the medium and try to create unique experiences. It's Beat Saber and Gorilla Tag, extremely simple games that didn't require a lot of effort, just luckily landed a viral hit, just ran with it avoiding major screw ups, making more than USD 100M each.

            It is of course frustrating for any developer that hard work and passion in no way guarantees (financial) success, or at least some recognition, so it is a lot more comfortable to believe in the VR version of the American dream. But the gaming business reality is very different, which is why mobile games using microtransactions rose to more than 50% of all gaming revenue in just a few years, why large studios spend hundreds of millions on the next live-service game only to scrap it if it doesn't go viral, or why Meta will go after the Gorilla Tag and Fortnite crowd with Horizon Worlds instead of those that believe that Baldur's Gate 3 irrefutably proved that in the end good content will win.

          • VRDeveloper

            I need to admit, you're right.. my argument really was biased. That's real life: they'll keep producing garbage, people will keep consuming garbage, they'll get progressively dumber, and VR will die again for anyone looking for good games it's already dead. That's it… part of life. The world will always be this shit.

            I just hope there are still people to buy my game.

          • Olle

            I disagree with this sentiment. I think it is pretentious to say that the things you don't like is garbage. Imagine a bunch of kids who doesn't have the same references as adults and who find these AI bots amazing to interact with. Who is anyone to judge them? I find grown-ups whos main interest is numbing reality through somewhat richer VR-apps alot easier to point finger at, tho I won't.

          • VRDeveloper

            If you believe this stuff isn’t garbage, and that it isn’t making our children dumber, then go ahead and keep consuming shallow content instead of works that actually carry messages and lessons. I don’t care anymore. I honestly hope society goes down the drain… I’ll just focus on those who value depth, and the rest can go to hell.

          • Christian Schildwaechter

            Every generation has complained that their kids are wasting their time with stupid media, claimed this would inevitably lead to society dumbing down, and considered teen speak an atrocious abuse of language. And every generation has forgotten that their parents said exactly the same things about them, and their grandparents the same things about their parents. And they were all wrong.

            We now have roughly half a century of people claiming that certain video games turn people either violent, or indifferent, or teach the wrong values, or turn them stupid or unfit for society. This started on the Atari VCS 2600, is still ongoing, and for some unfathomable reason even people that grew up with video games repeat the same nonsense, just with a new coat of paint. Despite there now also being decades of actual research showing that no, this is absolutely not the case, and society is not going down the drain just because some people for whatever reason have fun with certain video games/media that others don't like.

          • Peter vasseur

            Why does a work need to carry a message? Why does there have to be any message. Why can’t there just be stuff that fun and doesn’t require trying to push “ your message” on people.

            Starting to sound like a wokey, which is actually what has made gaming worse.

          • sfmike

            And the Horizon Worlds' team seemingly controlled completely be feminist wokeys are really hard to take. Can't wait to talk to that overweight chick in the photo and have her tell me what a bad man I am. It's only recently that Meta has enabled us to create avatars that actually look masculine and not have doughy nonbinary bodies and faces but from from the illustration they have provided the AI NPCs will be PC gay looking women. They will never learn.

          • Christian Schildwaechter

            TL;DR: just because it isn't going the way you want it to doesn't mean VR is going to die.

            I don't think VR is going to die, and people are going to continue to create great games for it. But what will probably end is Meta massively pumping money into the market or subsidizing projects like RE4VR or AW2, because they didn't lead to the growth they hoped for, and most current Quest users don't care about them.

            And Gorilla Tag isn't garbage, it just leans on emerging social gameplay with an uncommon arm swinging locomotion (apparently inspired by Raccoon Lagoon) instead of graphics or complex mechanics. I live next to a school, so I am not at all surprised that a very physical version of VR tag has become popular with a younger crowd, while older gamers basically complain that it is not as fancy, polished and comfortable as their expensive cars. I'm also pretty sure that playing catch doesn't make kids dumber, in fact it is a great physical activity during recess.

            VR enthusiasts keep asking Meta to do what they want, mostly paying for more high profile games and releasing HMDs with better specs. And look down on the now much larger group of Quest users that are fine with very simple games, as long as they are fun for them, with fun being mostly about hanging out with friends and an adrenalin/dopamine rush. Graphics or complex game play aren't really that important, as the mobile gaming market should have taught everyone by now.

            The ones making the mistake here are the enthusiasts. Not by asking for better games, but by confusing their own interests with what the platform needs to grow. No, Meta shouldn't cancel Horizon Worlds and give the money to indie studios instead, and the TikTok crowd shouldn't start to play AW2 or Ghost of Tabor when they are having a lot of fun running around wildly screaming with their friends in Gorilla Tag. Of course it sucks to be stuck in a niche with too few players to pay for AAA or even AA, but that is neither Meta's nor the Gorilla Tag player's fault.

            For developers this means they have to come to terms with VR not going mainstream anytime soon, and support from Meta, either financially or by promoting games, being reduced. This makes it harder to break even in the small VR market, so they very likely have to reduce the scope to keep the development costs in check. But that is no different to most small PC game developers. These have a much larger potential audience, but also a lot more competition, with Steam and itch_io getting hundreds of new releases every day.

            That doesn't stop people from realizing their own dream project, you just have to be aware that only very few will ever manage to make a living by creating (VR) games. If you need the money, you'll have to go where the money is, and in the future that may be (still Meta subsidized) Horizon Worlds ,or the TikTok crowd looking for a different type of game than you want to create.

            To even have a chance, game developers now basically have to run a heavy social media campaign right from the beginning of the development, to build an audience that might notice and buy their game on launch, so it could be picked up by an recommendation algorithm. Which is pretty much how Gorilla Tag got going, not by being particularly innovative or pretty, but by looking fun to play in thousands of TikTok videos that got more than 10 billion views by mid 2024. There are a ton of GDC talks on YouTube and in the GDC Vault about how to survive as game developer, most of which apply to VR developers. And unless Meta completely drops Quest, them focusing on Horizon Worlds is just another thing developers have to adapt to, but certainly not the end of VR.

          • VRDeveloper

            We have to follow the money." Man, I give up, you must be the most conformist person I’ve ever met in my entire life. This kind of thinking is exactly what has impoverished our Western culture so much. What people consume does indeed influence their brain and cognitive development, which is why there are IQ measurements, and why IQ scores keep dropping year after year.

            This is also why we elect people like Trump, Biden, Lula, Bolsonaro, Milei, completely stupid individuals who represent very well the current society IQ, which can only see the superficial in everything.

            While our IQ rate drops, and we teach people to consume garbage just because it "makes money," China is regulating its networks, their IQ rate keeps rising, the products they produce are better, the country’s infrastructure is infinitely superior, everything is improving. We are sinking precisely because of this mindset of chasing money without considering morality or long-term cognitive development.

            Anyway, our conversation has already made me migrate to Steam. You opened my eyes to understand that there’s no space for deep works within Meta Quest, and the other comments reinforce that the audience drawn there also doesn’t care about this vision of VR.
            Let’s leave this conversation here before it turns into an infinite thread. This conversation is so pessimistic for someone with a vision of something truly productive, something that teaches people and has technical quality, that it makes me want to give up on VR.

          • Peter vasseur

            Meta isn’t the only vr company in town. Your doom is to much gloom and not reality.

          • Peter vasseur

            Free to plays can suck it. No matter how much passion. They gimmick games that should all go the way of the dodo.

  • Meta has obviously not read the Mortality Doctrine

  • rabs

    Makes me remember when Zuck said in May he'll provide AI friends to lone people. We thought about Facebook, but it's coming to VR too.

    If those can make a good Waifu, it may be of interest to more people :D

    There will be social backlash, but he doesn't care as long a you tell everything to his chatbots so you get a well profiled advertising.

  • rabs

    It's not about me, I don't use their stuff.

    Meanwhile, in latest news « Meta created flirty chatbots of Taylor Swift, other celebrities without permission ».

    They know what's good for engagement, as expected…

  • Cragheart

    Facebook is annoying
    why wouldn't this be?

  • JB1968

    ”Wow. What a cool idea. Anything is possible, so let's just map out a path, and we can try to figure it out.”

  • sfmike

    Also I feel the farming and town/city building games and even most role playing games teach that you have to work and save resources wisely to get ahead, a lesson right there not taught in school.