This week Meta showed off a preview of its next-gen avatars during the Connect keynote. Frankly, they looked great! However, there’s… less than meets the eye here.

When Mark Zuckerberg appeared in Horizon during the Connect 2022 keynote sporting a next-gen Meta avatar with complete body tracking I was frankly blown away. Compared to the existing leg-less avatars that everyone appears as in Horizon, these new models not only had a full body, but also incredibly natural and movements with not a glitch in sight… it seemed almost too good to be true.

From what I know of the existing tech and limitations for tracking the body from a headset… what was shown here would have amounted to breakthrough work from Meta. Here was Zuckerberg literally jumping around and touching his feet with seemingly perfect tracking. His on-stage guest also highlighted what appeared to be perfectly accurate elbow tracking, something that’s only roughly estimated with today’s avatars. Zuckerberg then proceeded to say that these avatars would be launching sometime next year.

But all is not what it seems. After reaching out to Meta to clarify what tech was used to create that segment, a spokesperson told me the team used common external motion capture tech to drive the movement of the avatars, rather than its own solution running real-time on one of its headsets.

From my conversation with Meta, the company explained the segment was intended to show off how the next-gen Meta avatars look (from a model and texture standpoint) but how they moved was, “presented for illustrative, forward-looking purposes,” the company said.

So I have serious doubts that the first full-body VR avatars from Meta will have this level of high quality body tracking—and I’d argue the company was a little deceptive here.

SEE ALSO
'Metal Gear Solid' Creator Teases Interest in VR with Virtual Booth at TGS

What we’re probably going to see in the first iteration of Meta avatars with legs is an AI inference of leg positions that will look plausible from a third-person view… but nothing like being able to jump around and touch your feet—in fact, from a first-person view you probably won’t be shown your virtual legs.

But there’s no doubt that Meta’s avatars have been getting consistently better and that the company has been doing tons of R&D to try to reach the vision Zuckerberg presented.

Meta is burning the candle on both ends here. For one it’s steadily upgrading the avatars that actually exist today—like new face-tracking that will come thanks to Quest Pro—to make them more expressive and human. And on the other end, the company is working to try to make the most photorealistic avatars it can with its Codec Avatar project, which it hopes will one day be usable even on low-powered headsets.

So I won’t say that Meta will never have avatars that show the kind of full-body tracking quality that was shown off at Connect 2022. In fact there’s an interesting possibility that VR controllers with on-board cameras (like Touch Pro) might be able to solve this problem one day, but it seems certain this won’t be the case for the first version of Meta’s legged avatars.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • Ah

  • ViRGiN

    I think publishing the response that you’ve got from Meta would be more beneficial for context, rather than your own interpretation.

    > the team used common external motion capture tech to drive the movement of the avatars, rather than its own solution running real-time on one of its headsets
    That does sound like they have body tracking in some early stages, but it’s hard to know without exact quote.

    Nonetheless, that’s a big bummer and very misleading.
    But it’s not really different than gabe newell from valve lies.

    > “Wireless is a solved problem at this point,” Valve founder Gabe Newell told a roundtable of developers at the company’s headquarters in Bellevue, Wash. “So, my expectation is that it will be an add-on in 2017, and it will be an integrated feature in 2018.”

    Almost 2023, and there is not a single solution available for their own headset. Don’t even get started on Nofio lol. They got half a million, with shipping in April 2023 which obviously will never happen. Arpara kickstarter got locked out of Kickstarter for not sticking to their promises and ignoring customers.

    • Jeff is beck

      Cope

      • ViRGiN

        can’t cope, too busy buying 5 kilograms of games for 5 dollars on steam vr fest

        • I know you’re anti-Valve, and there’s legitimacy there absolutely.
          But I’m super-curious about what your thoughts on Deckard are …?

          • ViRGiN

            Non-existent. Deckard is around the corner for over a year now.
            When it’s finally out, it will be driven likely by XR2-some-variant, and that’s when PCVR elitists will finally accept mobile VR to their homes.
            Much to their surprise that games aren’t crossbuy, and now they will have to purchase Half-Life Alyx: The Mobile Edition, but that will be fine, as game development costs a lot of money and they just want to support VR industry, unlike the years prior.

          • Ookami

            Yes that makes total sense, seeing how Valve forced everyone to buy mobile versions of games to play it on the Steam Deck–Oh wait: they didn’t

          • Gabe Zuckerwell

            Valve doesn’t sell Deck games. Nice try, gaben.

          • Ookami

            Critical error:
            brain process failed

            Retry? Cancel?

          • Gabe Zuckerwell

            Cope

          • Ookami

            lol

          • Sven Viking

            Not sure about their other games but they sell Half-Life 2 and Portal 2 as “Deck-verified” games.

          • Gabe Zuckerwell

            Deck Verified isn’t the same as Made For Deck. It runs exactly the same files as standard PC.
            Deckard isn’t going to be mobile pc running Windows apps.

          • Sven Viking

            Ah didn’t realise you were meaning nobody sells games for Deck.

          • Cl

            He’s just upset that not everyone likes the things he likes. I honestly can’t recall anyone saying Deckard was around the corner. Just excitement for it and wanting more info.

          • ViRGiN

            ah, yes nobody. sounds like you are just upset you still have to play rec room and beat saber.

          • kontis

            Deckard is not gonna be what Brad hopes it is.

            It absolutely CANNOT be a crazy high end HMD with uOLED and pancake lenses because it would cost $2K and Steam wants to sell a lot of games, not just a few. It will be LCD based.

            Gaben has a Quest 2 envy, because it makes bigger numbers than SteamVR. And that thig is relatively cheap.

            Steam Deck being low cost oriented explains everything.

            Something like Pico 4 but with full Linux is the best hope. x86 emulation would be a cherry on top (Beat Saber etc.) and there are 2 teams working hard on it, one even has Valve devs on their discord…

            tl;dr Deckard will directly rival Quest 3, not Quest PRO or Apple HMD.

          • Christian Schildwaechter

            It still seems unlikely that Deckard would use an ARM SoC as its main processor, when Valve just release a gaming handheld based on a very power efficient AMD x86 APU that allows to run most of the existing Steam Library and already said that the technology would be usable for VR. Apple managed to get x86 emulation speeds en par to their previous Intel based Macs, which was quite an achievement and required extending the ARM instruction set on there CPUs, but x86 apps are still way slower on an M1/M2 than the same app recompiled.

            Emulation is okay for running old software that otherwise wouldn’t be usable anymore, but it is a very bad idea for VR apps on mobile. So picking anything other than an x86 SoC for Deckard would be Valve throwing away all of the advances they have over e.g. the Quest, incl. Windows compatibility for games on a Linux variant highly optimized for Gaming already running on a tiny mainboard the size of a pack of cigarettes.

            The main advantages of ARM over x86 on mobile were lower/faster power saving modes, allowing phones to quickly switch between burst and almost sleep mode to improve battery life. Meta and others went with ARM because a lot of tech from phones that are technically very similar to mobile HMDs already existed. Their first mobile VR venture was the Samsung Note 4 based Gear VR, so the architecture is at least somewhat inherited, not the only reasonable choice. Hololens and Magic Leap 2 already use x86, and we know that AMD’s roadmap will bring us fully VR capable low power x86 APUs.

            I’d agree that Valve will consider the Quest 3 more as a rival than the Quest Pro, with an HMD designed with low production costs in mind, and I’d expect it to be conceptually similar to the Steam Deck: a very portable version of the devices that already exist, running the current Steam library. Which is the source of all Valve income, allowing them to sell cheap hardware to users that could then chose from currently 6,810 VR supported titles on Steam. If fast enough, an x86 Deckard could even run (some of) the PCVR mods of existing pancake games.

          • Mick3

            There are already people running SteamVR on the Steam Deck using the LG VR 360 smartphone glasses via USB-C.

            The rather low resolution of the LG headset (2x 720px x 960px) fits nicely to limited 3D performance of the console GPU.
            6 DOF tracking and controller support obviously needs some additional hardware.

          • Christian Schildwaechter

            It is also possible to stream VR wireless from the Steam Deck to a Quest 2, with full 6DoF and controller support, but this currently requires installing Windows 10, as neither the Oculus software nor VirtualDesktop on SteamVR run under Linux/Proton. It works well for less demanding titles like Beat Saber or Pistol Whip. It is even possible to play HL:A at mostly stable 60Hz, at least the earlier parts of the game, by setting VD to potato mode with SSW reprojection and reduced render resolution. And that is without any of the configuration hacks that people used to get HL:A to run on weaker PCs and on a definitely not optimal setup.

            This could probably be optimized a lot, but so far this is rather academic, as the Steam Deck GPU provides about 30% less performance than a VR entry level GTX 1060. It is an interesting demonstration how powerful a SoC with a power envelope of 15W for CPU and GPU combined can be, which is one of the reasons why I’m pretty sure that Deckard will use a future version of the AMD Van Gogh APU, but the extra hassle required and the subpar experience limit the practical use of the current Steam Deck for VR.

          • Mick3

            I think the difference is larger than 30%. The GTX 1060 does ca. 4 TFLOPS FP32 calculations using 200 GB/s memory bandwith compared to 1.6 GFLOPS and 88 GB/s of Steam Deck.

            Considering the better efficiency of the RDNA2 GPU compared to Turing then it’s more or less still a factor of two.

          • Christian Schildwaechter

            Corrected. The “30% less performance” should have been a reference to the GTX 1050Ti, which became the VR entry level GPU according to Oculus after they introduced ASW. It is hard to find useful benchmarks that also run on Linux, making it difficult to evaluate average game performance, as so much depends on things like drivers, proton support or different system architecture with shared memory.

            The Steam Deck allows allocating CPU and GPU power according to user settings, and The Phawx showed that you can improve performance in some games by manually disabling CPU cores due to the resulting improved total power use, so actual game performance will also depend on different system settings. I’d expect Valve to tune the system/OS for the specific needs of VR in Deckard, just like they did with SteamOS for pancake games, casting more doubts on projections based on the Steam Deck besides “with potential, but currently too slow”.

          • Mick3

            You can use ALVR on the Quest to stream SteamVR apps from Linux. It should work the same way as with Windows.
            The SideQuest version seems a bit old so better download the latest release from Github.

            I have no idea how good the Proton support for Windows SteamVR binaries via Steam OpenVR on Linux is but maybe I’ll try to find it out some day.

            At least Valve’s and other’s native Linux VR games like HL:Alyx should run and get streamed via ALVR.

            > It is also possible to stream VR wireless from the Steam Deck
            > to a Quest 2, with full 6DoF and controller support, but this
            > currently requires installing Windows 10, as neither the Oculus
            > software nor VirtualDesktop on SteamVR run under
            > Linux/Proton.

          • Christian Schildwaechter

            I got my Steam Deck with the last Q1 batch and immediately looked into ALVR, but in the end didn’t try it. Back then it just had gotten experimental support for Linux, but the bigger problem was that it didn’t seem to support any type of reprojection like ASW on Link or SSW on VirtualDesktop, which is kind of essential due to the slow GPU. In theory SteamVR motion smoothing should be available, but motion smoothing on Linux is/was only available with Nvidia graphics drivers, so not an option on the AMD Steam Deck.

            ALVR only got two new releases since February, I didn’t check the nightly builds. The general consensus seems to be that it (barely) works on the Steam Deck, but so far isn’t really useful, though I haven’t tried myself yet. In theory ALVR being open source could provide the best solution and allow for a lot of optimization potential, but right now it is like all VR on Steam Deck: you can get it to work, but you need to make a lot of effort, and the result isn’t great.

            The Steam Deck as a mobile handheld running Windows games on Linux actually works great thanks to all the support from Valve, and this includes running VR games via OpenVR on Linux. ProtonDB lists a lot of VR games with platinum support, running mostly flawless without needing any tweaks. Technically you could run Windows games under Linux for years, Proton is a gaming optimized version of Wine, which has been around for decades. But it wasn’t exactly a smooth ride, so I wouldn’t have recommended it to anybody unless their interest was mostly tweaking systems to get things to run.

            This user group definitely exists, for example on Steam Deck lots of people seem to spend most of their time setting up and tweaking dozens of emulators, only to then barely use them. I’d say that VR on Steam Deck is currently similar: a hobby involving the challenge of getting it to run mostly for the fun of the challenge itself. Those interested in just using VR on mobile SteamOS would be better served by waiting until Valve releases Deckard and will have solved a lot of the problems, the results of which will also become available on other platforms thanks to the open source base.

          • ViRGiN

            Oh wow. A windows pc running windows software? COLOR ME SURPRISED! WHO WOULD HAVE EVER THOUGH A WINDOWS MACHINE POWERED BY BATTERY COULD EVER RUN STEAMVR?

          • Talking of Deckard, check out Brad’s twitter today.
            A new Valve game franchise has been discovered.
            It’s called “Neon Prime”, are guesses are that it’s acyberpunk title.

          • ViRGiN

            >A new Valve game franchise
            WOW! MIND NOT BLOWN!
            is this the sixth or seventh game discovered?

            Or maybe it’s yet another handshake simulator?

          • Cl

            Oculus already drives steam games sales. So they can make a really high end one since the low end is already covered. I can see them focusing on a nice one focused on pcvr. I don’t see them making a direct competitor to quest3, but who knows

    • kontis

      But it’s not really different than gabe newell from valve lies.

      WRONG. You of course are insanely biased, as always.
      Gaben was wrong, not the first time. He made many bad predictions over the years (the infamous AppleTV destroying PlayStation is my favorite one).
      However, saying something incorrect in a hour long interview is INCOMPARABLE to a huge PR camping targeted at investors with a whole FAKED live demo with the actual CEO on stage.

      The difference is astronomical.

      • ViRGiN

        Obvious valve shill is obvious valve shill.

    • Ookami

      “But Vaaaaalve!”

      Copium

      • xyzs

        No real legs for Meta, no braincells for virginity. That’s probably Gabe’s fault in the end…

        • Ben said tone down on the personal insult stuff.
          Tread carefully, my misguided friend ….

          • xyzs

            Well if Ben cared about respect that much, he would have banned Virgin the troll for a long time…

  • Rupert Jung

    “Fake” needs less letters and also describes this presentation accordingly. They used pre-recorded, external motion tracking and presented it in a way to make the audience believe in a realtime solution available right now or in the near future. This is almost Magic Leap level of dishonesty.

    • kontis

      It’s even worse than that.
      Zuck while faking the demo on stage also lied that none of the competitors could do it. Some of the people watching it using FBT tracking for years must have been dumbfounded…

      • Aw MAN, and he made it look so cool …. lol
        I watched Meta Connnect in VR on my Quest 2.
        We were all seated in an auditorium that changed themes
        as different things were talked about.

        Suddenly, the auditorium transforms into a stage when Zuck
        and the girl both come out, and me & the rest of the audience
        spontaneously rushed the stage’s edge like at a Depeche Mode concert! lol

        • CURTROCK

          I was there…It was a real METAVERSE moment. The transforming stage sets, HD video, and then the avatars coming out. This really showed the true potential of Horizon – it all came together perfectly.

      • XRC

        Looking at my Vive trackers here…

      • ViRGiN

        FBT? You mean kinect people you valve shill?
        “vive trackers” are supported by like “5 games” which aren’t even games, and they do not track the body – they track the tracker itself.
        Been waiting for over 2 years now for someone to finally link me up to Body Tracking SDK. There is none you idiots. You can attach vive tracker to a stray cat and boom, you got yourself a CAT TRACKER.

        • kool

          I dont think your get to use shill tho lol.

          • ViRGiN

            i said, therefore it’s truth.
            works the same way in opposite direction.
            it’s never about calling the truth, it’s always about calling names.

          • kool

            You didn’t lie, but you’ve sold more than a few quest on here lol. Nobody has released body tracking yet, im surprised nobody has shown off any metaverse games just lame lobby stuff.

          • ViRGiN

            I’ve sold zero quests, especially here on RTVR. Quest sells on itself, and thanks to actual content. My comments influence no one.

            Look into body tracking technologies itself – companies like rokoko and xsens selling ~5000 dollars gyroscope crap. next to nobody actually uses it, and the rare occasions you see anything on youtube is just ‘testing’ stuff, and the few under a minute long mocap videos doesnt even look like were captured this way.

            body tracking is not a solved technology for even high paying enthusiasts. vive trackers likely are the most stable ‘solution’, but it’s still a joke and essentially nothing more than a controller glued to legs etc. yes you do can communicate lifting up your left leg, but that’s not body tracking and it’s not convincing at all. funny how all these vrchat abusers think they are ahead, in the future.

        • Cl

          Oh look, something valve does that Facebook can’t yet. It’s one of the benefits of that type of tracking, but your brain can’t accept there are any benefits.

          • ViRGiN

            hey fool, vive trackers arent even from valve. nice valve shilling you got out there! i bet you get paid in steam-points to spend it on steam-emojis.

      • Dave

        Come on you are just being silly. Mark is talking about full body tracking from the HMD/Controllers. Of course if you add
        FBT devices to every joint on the body you’ll eventually get FBT.

    • Dave

      How do you know for sure it’s fake. Maybe they use controller cameras as well with the headset to build a picture of the FBT. They’ve shown strong results of side by side body tracking (human verses simulated) in past Connects.

      • Sven Viking

        “After reaching out to Meta to clarify what tech was used to create that segment, a spokesperson told me the team used common external motion capture tech to drive the movement of the avatars, rather than its own solution running real-time on one of its headsets.”

        • Dave

          Ah ok, I’ve seen the full presentation now, not just Mark jumping and yes it does look like something not captured from the HMD. Thank you for the clarification. That was a bit naughty by Meta. Oh dear.

          • ViRGiN

            Once it’s done, i expect it to be tracked by controllers sitting on the dock station.

  • I mean Meta Connect *2*.

  • Sandrine

    Really?!

  • ViRGiN

    The time to get excited about deckard is when they publish a kickstarter campaign. Valve is too broken to risk such a risky move. They do not care about vr, so why should i care about them?

  • ViRGiN

    always has been. xyzs is disagreeing with me, OBVIOUSLY he is THE TROLL here.

  • Craig Duncan

    The Pico 4 seems to have Motion Trackers to put on your legs that seems to address this. I don’t know why Meta care enough to fake the solution but don’t provide a solution like this?

    • Ben Lang

      Not quite. Their ‘motion trackers’ are just 3DOF sensors for more accurate fitness estimation. My understanding is that they aren’t planning to use these for leg tracking, but maybe they’ll work them into their IK model in the future.

  • Kenny Thompson

    Its crazy…. Everyone from Sony to Microsoft have been BSing tech deoms for DECADES. Yet FB makes a video that will come true in short order yet people jump outta trees to harass them. Remember the 1m tri talking head from the PS2 era? The Girl / Robot from xBox? The Syphon Filter PS3 demo?