Much ado has been made about Meta’s latest avatars which juxtapose impressive expression against, well… a complete lack of legs. While full-body tracking is both desirable and achievable today with outside-in tracking systems, Meta doesn’t think it’s viable with inside-out tracking on headsets like Quest 2. However, the company say it’s investigating ‘fake’ legs instead.

Andrew “Boz” Bosworth is the VP of Meta Reality Labs (the company’s XR division), and soon to be CTO of Meta overall. In his role heading the company’s XR division he’s made a habit of doing impromptu Q&A sessions via Instagram where he answers both personal and work-related questions.

In his latest Q&A he was asked about the potential for full-body tracking on future headsets from the company, but dismissed the idea as not viable with inside-out tracking.

While Quest 2 is presently capable of head and hand-tracking (which makes it relatively easy to estimate the position of the arms and chest too), the headset has no concept of where your legs, feet, or hips are, and that’s why Meta’s avatars are essentially chopped off below the waist when you see them in VR.

Image courtesy Meta

On the other hand, other VR systems using cameras external to the headset (known as outside-in tracking) are capable of full-body tracking to enable more lifelike avatars and some use-cases that simply wouldn’t be possible without it.

It’s been suggested that with some computer-vision trickery, perhaps tracking cameras mounted on the headset (known as inside-out tracking) could be used to estimate the position of the user’s feet. Bosworth reasoned that this is not only extremely difficult given the position of the cameras on Quest 2, but will become even harder as headsets shrink.

Body tracking is super tricky. Because from the camera that’s on your face it can’t see your legs very well. And as we want to make the [headset] smaller form-factor, it gets even worse—[the cameras] can’t even see past your cheek sometimes to your upper body. Now we can get away pretty well with the upper body because we can see your arms, elbows, hands, and we kind of have a sense of what the musculoskeletal structure must be doing behind it, but feet are tough.

So [using outside-in tracking as opposed to inside-out] is probably necessary for some of the [full-body tracking] use-cases people have in mind. So that’s one of the things that we’re looking at.

Now there’s been a leaked thing [about full-body tracking on Quest]… I think that’s actually probably a little premature… there’s nothing substantial behind that necessarily. But it’s something that we’re always looking at and have in mind for sure.

The “leaked thing” Bosworth mentioned is likely a hint about body tracking in Quest 2 documentation that was spotted by our friends over at UploadVR.

Later in the Q&A, Bosworth again addressed Meta’s legless avatars and suggested that the company will probably move forward with ‘fake’ legs, which would only be seen from a third-person vantage point, as a stopgap.

Tracking your own legs accurately is super hard and basically not workable just from a physics standpoint with existing headsets [that use inside-out tracking]. You could go outside-in body tracking—but that’s an extra component, an extra cost, an extra setup—it’s a lot. Or you could have no legs, but everyone else when they look at you could see that you have legs and we could fake it, and no one would know the difference. That’s a better direction.

Obviously we’ve read the jokes [about our legless avatars], we appreciate them; they are very funny and very fair. So we are looking at how we can do [some form of legs]… if you look at your own legs and you see them out of position, that is a very bad experience and you feel very dysmorphic, but if you look at someone else’s legs and we just made up their position but they seemed reasonable, you’d just be like ‘yeah that’s probably where [their legs] are.’

Naturally Bosworth understands that the fake legs approach wouldn’t enable use-cases like using your legs as game input or for things like dancing, but at least in simple social VR settings—where avatars are just hanging out and chatting—it would be nice to have avatars that aren’t awkwardly missing their lower half while floating in the air.

SEE ALSO
Meta Planning Web-based Version of ‘Horizon Worlds’ With Lower Creator Fees Than Its VR Platform

Elsewhere in the Q&A Bosworth was asked if Meta would consider subscription-based access to the Quest content library (akin to Xbox Game Pass or Viveport Infinity). He said he didn’t think the company “has a catalog yet that could sustain a subscription,” further saying that cross-platform content would probably also be necessary for that business model to work (which is not presently the case with the Quest store). He did indicate that the company is likely to introduce gift cards for the Quest store in the future.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • xyzs

    When was the latest good news with Meta ?

    Cambria reveal cancelled.
    Cambria release delayed.
    Cambria might ditch OLED for mini led LCD
    Quest 3 not released before at least 2023.
    PC VR getting abandoned

    • Blaexe

      “Cambria reveal cancelled.
      Cambria release delayed.”

      That’s not true?

      “Quest 3 not released before at least 2023.”

      How is that “bad news” (it’s not even news since we have nothing official). Seems like a decent plan and I guessed Quest 3 wouldn’t release before 2023.

      Also this article isn’t news either. It’s a question within an AMA answered. And the answer makes sense.

      • Daniel Lingard Leaper

        I think Quest 3 not releasing until after PSVR2 is a bad thing because they’re going to lose a lot of users to PS as the HMD is better in every way and most importantly can deliver visuals Quest cannot match. I don’t see many that move to PSVR2 ever coming back to Meta unless the Quest 3 had tech that is not yet know and by the sounds of it will only be a lighter version of the Cambria to keep costs down meaning pretty much all its ‘new’ features will just be features the PSVR2 has……but with shitter graphics. I’ve been with the Quest since the beginning but as soon as PSVR2 comes I’ll never go back, and if PS come up with a wireless solution in the future & xbox release a HMD……Meta are done, this gamble looks ever more risky to me. I mean Meta have spent what 10b to gain maybe 6m users, how much do you think its cost PS? I’m guessing nowhere near that amount & over night they’ll probably have as many users. For me I think this is a massive massive mistake by Zuckerberg, someone sold him an idea & hes so deep now all he can do is double down on it.

        • Blaexe

          Quest and PSVR2 are not even direct competitors. One costs $300. The other costs somewhere between $800 to $1000 to use it.

          Gamers and VR fans with disposable income will get both anyway.

          There will be a lot more Quests sold than PSVR2.

          • Daniel Lingard Leaper

            PSVR sold about 5m units at a higher price than Quest with controllers tracked by 10 year old repurposed tech & little fanfare or support, Quest has sold maybe 6m…………..I think you might want to rethink it a bit.

          • Blaexe

            PSVR sold 5m units within a bit more than 3 years. Quest 2 sold around 10m units within a bit more than 1 year. That’s 6x faster.

            PSVR had an attach rate of around 5%, so 5% of all PS4 owners had a PSVR.

            Quest 2 + Quest 3 will sell far more units than PSVR2.

          • Cragheart

            5 million reported in 2019

        • Zigmar

          While PSVR2 might have better hardware and will have a better graphics, it is still a tethered VR, which is two steps back in terms of usability and experience. I own a powerful gaming PC and couple of PCVR headsets, yet 95% of my VR usage is on Quest2 in standalone mode. For me and many other VR users, convenience, ergonomics and accessibility is far more important than a slightly better graphics. Other people might have other priorities, but I don’t expect PSVR2 to take a significant market share of VR market.

          • Daniel Lingard Leaper

            Really not sure how you can say it wont take a significant market share, for 1 those 10m sales for Q2 are bollocks, any industry insider will tell you the figure is more like 6m maybe 7m at best. Secondly PSVR sold 5m units with a shit product from about 5 years ago that cost £100 more, it had a wire, none of that stopped 5m people buying it.
            The wire is a barrier but slightly better graphics? are you kidding, I don’t know what PCVR games you’ve played but slightly better?There were 116m PS4’s sold, I’d imagine eventually that number will be smashed by PS5, add to that the greater interest in VR in general I’ll be amazed if they don’t sell 10m PSVR’s over a period of 3 – 5 years. The only thing Quest has going for it is the lack of a wire, which admittedly is a massive plus but I own a Quest & I’m buying a PSVR2 & I know a lot of other people who will, then there’s all the people who don’t have Quests but will buy a PS5 & give VR a try through that.

          • Blaexe

            > any industry insider will tell you the figure is more like 6m maybe 7m at best

            Industry leaders like Qualcomm?

            > I’ll be amazed if they don’t sell 10m PSVR’s over a period of 3 – 5 years

            And the Quest line will be in the 50m range by then.

          • Daniel Lingard Leaper

            I meant industry insiders that don’t have a linked motive to tell you how great they are. Plus Qualcomm haven’t said there have been 10m Quests sold. Unless Meta release figures its a guess, & people who aren’t intrinsically linked to Meta don’t believe there are 10m Quests.
            They wont have 50m in 3 – 5 years ya more than optimistic there. What you’re not factoring in is that a shit load of the people who bought a Quest because it is so cheap & therefore low risk…………will not be buying a Quest 3! I have about 30 Friends on Quest through various apps, out of all those people about 5 use it at least once a week, maybe 10 turn it on once a month, and the rest have never been online for a year or more. A lot of people who try VR do so for a few weeks or a month or 2 and never appear again………I mean never! The wow factor disappears & it gets put in a cupboard, all those people will not be buying a Quest 3.
            In regard to the PSVR2 sales, I’m more than happy for you to bookmark this thread in your mind, remember my name & come back to me in a couple of years & either show me that the PSVR2 isnt one of the most used VR HMD’s on the market or that it is. Take care.

          • Zigmar

            PSVR numbers are pretty much irrelevant. When PSVR came out every VR headset was tethered. And PSVR was a good great l headset and cheaper option in case you didn’t have a PC but owned PS. Things are different now, when you can buy standalone headset for a fraction of the price of tethered PSVR2+PS5. Sure, it probably will be pretty popular among people who already open PS5 but the numbers would barely scratching Quest2 which targets much much bigger market. I see people buying Quests for children and elderly parents. I see people buying multiple Quests on same household to play multiplayer games like 11TableTennis. To all those people PSVR is pretty much irrelevant, they want something that is easy to use and cheap. They aren’t going to buy PS5 when all they want is to play Table Tennis or do a workout in VR.

          • Cless

            You own a “powerful PC” and multiple PCVR headsets… you are clearly enticed by the wire free that Quest 2 gives… and instead of getting a wireless mod for your PCVR headsets… you get a Quest 2? interesting…

          • Zigmar

            I do sometimes play wireless PCVR. But it is just not as smooth and simple as naive Quest app, where I can just pop the headset and be in a game within seconds.

          • Cragheart

            Slightly? Are you sure?

          • Zigmar

            No matter how better the graphics are, they are not good enough *for me* to go back to tethered VR

      • xyzs

        Yes reveal was cancelled. In their lastest lame metaverse video, some parts where Cambria is shown were clearly removed from the cut.

        Quest 3 being late is a bad news: VR is not like flat gaming. Tech is not polished and stable for many years, there is a need for strong and fast iterations/innovation since VR is its infancy and since its technical limitations keep it from being mainstream enough to be profitable. VR cannot have the same progress pace as established consoles right now! It needs yearly hardware updates to reach the specs it needs, just like what happened with smarphones.

        • Blaexe

          Cambria literally was revealed. Otherwise we wouldn’t know about it.

          I think Quest 3 coming 2023 is a very good thing. It tells buyers that they’ll get decent value out of it – Quest 2 will likely be on the market for 3 years and will probably be supported for 4 years.

          Also that way we can expect a pretty good technical jump – Micro OLED, Pancake lenses, face tracking, eye tracking…

          A Quest 3 this your would be significantly worse technically.

          PSVR2 might be on the market for 6+ years…

        • overzeetop

          There’s actually been quite a bit of progress using the existing hardware in the Q2 since launch – increase in frame rates, OEM supported wireless PCVR, hand tracking among them). The core advances in processing power, display technology, and power efficiency of 10-15% a year simply don’t justify new hardware releases (and the hardware/sku fragmentation it causes).

          • Cragheart

            Recent smartphone SoC improvements were so small, that Quest 3 can’t happen this year. It needs a serious hardware upgrade.

        • Cragheart

          Quest 2 is super slow, but Quest 3 with only 40-50% faster specs wouldn’t change much. That’s the problem. I agree that VR sucks right now and it needs a lot of improvements, but smartphone progress has very much slowed down. Smartphones used to double performance every year until they didn’t. RAM used to double every 2 years until it stopped. Chip shortage makes things additionally problematic. From what I see, VR will develop very slowly unfortunately. Even in 2032, standalone headsets won’t provide performance of current high-end PCs. That’s why it’s better to stick with PC VR or perhaps PS VR2. Mobile VR is shiet honestly. :-/

    • ViRGiN

      You forgot to mention Valve and their laziness since 2016.

    • Cragheart

      Life is full of disappointments.

  • kontis

    Good to know this most advanced XR megacorporation pumping 10 Billion into VR research finally figured out the basics that Valve, HTC and VRChat knew in 2016

    ;-)

    (just kidding, I know he simply answered a question and explained the problem)

    • ViRGiN

      To this day Valve, SteamVR, HTC, Tundra, or whatever you want to call them, does not support body tracking. At best, it’s just an external tracker, and then you have all sort of amateur software to translate that data into something that mimicks body tracking.

  • Jonathan Winters III

    Legs: great idea and about time.

  • It’s fair… good point about shrinking headsets making feet tracking even harder

    • Tommy

      Especially if you have a beer gut like me

  • VR5

    My guess still is that Project Cambria’s self tracking controllers* could be used for outside-in body tracking.
    The mirror in the leaked video was interpreted by some to utilise an actual mirror but you wouldn’t need a PC for that, a regular Quest should already be able to track a mirror image, unless the color camera somehow makes body tracking more viable (I don’t see how it would).
    Basically when the controllers are on the charging station, their cameras could track the user’s body and provide a virtual mirror. If you would put the second controller behind the user, it could even track them from all sides.

    *which have their own inside-out tracking instead of the outside-in on the Quest controllers.

    • Blaexe

      Computervision based full body tracking is computational expensive, it’s not an easy issue to solve. I very much doubt the controllers (with minimal computing power to save battery life) will be able to do that.

      • VR5

        The controllers already have to track the environment, just like the headset does. Supposedly different camera nodes can share their geometrical analysis on the snapdragon XR chip in a kind of cloud network since they are interpreting the same environment.

        A full body is more complex to track than surfaces but at least for hand tracking, it already works. The controllers can probably capture a small data size preanalysis* of what they see which the headset then finishes the body construction for.

        *Which is quick to transmit over wireless -> low latency.

        • Blaexe

          SLAM tracking has been solved for quite a while – unlike body tracking. Also there are rumors that the headset sends out a distinct pattern for the controllers to track.

          You can bet that the controllers will have a very minimal SoC, so I’m not sure the comparison with stuff running on the HMD is valid.

          • VR5

            While on the charging station they could draw more power than when used as controllers. It might also not be a problem since they work in a network with the other SoCs on the HMD and the other controller.

            Question is, can the XR chip do body tracking? If it does, the controllers being lesser power isn’t a problem. The leaked video does show body tracking, implying that Cambria can do it. The cameras on the controllers would provide the feed, preprocessed or raw, it would explain how Cambria provides this functionality.

          • Blaexe

            The leaked video only kind of implies body tracking in front of a mirror. Sending the video feed over from the controllers to the headset with minimal latency is not possible, the calculations would have to happen on the controllers.

            I’m sceptical. Let’s talk again in half a year.

          • VR5

            If they use a raw video feed that would introduce latency, yes. But since the controllers can self track, they should be able to have some already abstracted feed of preliminary geometry. The controllers just wouldn’t be able to tell which part of the geometry is the body. That’s a further step which would happen on the HMD.

            The mirror is a result of it being a front facing setup with both controllers on the charging station. It’s the same as Rift with two sensors on the sides of your monitor, or PSVR with one camera on the TV. You need at least one more camera behind the player for 360 tracking.

    • It’s also worth pointing out that if Facebook have figured out robust positional tracking self-contained on the controller, then you’re not too far off from something akin to the Vive or tundra trackers just using inside out tracking.

    • Tommy

      I think the cameras are on top of the controllers. Doesn’t sound like it would be able to track your legs

      • VR5

        Going by leaked pictures, each controller has three cameras, one on the plate with the face buttons and stick, which faces upwards, and two more on the front. Depending how you orient the base station, those two would either face away from the room with the player, making you save from being filmed without permission, or facing towards it. In the latter case the angle should allow to capture the full body, if the player is far away enough.

  • Sven Viking

    So likely the body tracking SDK settings are either for arms/elbows only, or for some form of optional external camera tracking.

  • Keith James

    I dont mind if I have to strap a few bands to my legs to get them tracked for stand alone (not valve type camera trackers) VR, but I want it soon! I love kickboxing, and cant stand that the Thrill of the Fight boxers dont feel my kicks…

  • Cl

    Maybe the new controllers with the cameras on it can help

  • namekuseijin

    aside from dancing crowds, I don’t see any benefit from full body tracking. it’ll be actually a nightmare when VR indies decide you don’t need to skip leg day anymore in their fitness minigames…

  • Lulu Vi Britannia

    If they pull off self-tracked controllers (as they’re apparently trying to do with Cambria), they can 100% do Full Body tracking at least for PCVR inside-out tracking. It would just require a few “self-tracked trackers” strapped to the feet. Not the end of the world. Of course it would cost more, but as long as it’s an add-on, not mandatory for a proper VR experience, there’s nothing wrong with it.

    Now, of course the price shouldn’t be 100$ per tracker, like Valve did a few years ago… But Oculus has always been able to sell at a fair price, so they could very well make trackers that can be used to sync legs to VR legs…

    • Tommy

      Only if they have down facing cameras. The prototypes I have seen all have the cameras on top of the controllers. They wouldn’t track your legs

      • Lulu Vi Britannia

        Not necessarily. To be clear, I’m not talking about cameras watching the legs here. i’m talking about cameras watching the walls. The ones on the headset track the headset, the ones on the controller tracks the controller, and the ones of the legs track the legs. They don’t need to be facing any direction whatsoever, as they just need to view the room.

        • Tommy

          How does the headset see the trackers on the legs though? I get the controller tracking with the walls but still confused as to what sees the legs

          • Lulu Vi Britannia

            Do you see how the Quest works? It’s got cameras, but these don’t see your head, right? And yet it does track your head. That’s because the cameras see the walls of your room, pinpoint your head location relative to each of these walls, thus getting the accurate location of your head in the space. That’s the way the Quest headset is tracked, and that’s how the Cambria controllers will be working too : instead of being watched by cameras, those controllers look at the walls, pinpoint their own location just like the headset.
            Well, the Leg trackers would work the same. Say you’ve got trackers attached to your ankle. The cameras on the trackers work just like the ones on the headset. They can pinpoint the exact location of the tracker attached to your ankle, therefore, they can pinpoint your ankle. So, no need to see the legs themselves ^^. The software will know this tracker is attached to your ankle, that’s how SteamVR trackers work too.

            The point is, a VR tracker works the exact same way as a VR controller. You can already make the experience : attach your Quest headset to your knee, and move. The location of your Quest headset in-game moves according to the motions of your knee, thus, it tracks your knee. Of course the headset is too big for that, lol, but a tracker would work the same way, see what I mean?

            Now, if you’re wondering how it sees the entire leg : well, it doesn’t need to. With a controller in your hand, a VR system can give you arms, right? They’re not in the exact position as your own arms, but they’re still effective arms in VR. That’s the same for the legs: you only need a tracker attached to your ankle, and you’re good for useful Leg tracking. But if you want more accurate tracking, you can just use several trackers : one for the ankle, one for the knee, one for the hip. (Same for the arm : we could attach a tracker to each elbow, getting exact Arm tracking). Each articulation of your leg is located individually.

            That’s why the only real barrier to Full Body Tracking is the price. Accurate FBT will require several trackers. But the tracking itself, in terms of technology, Oculus can already pull it off.

            (Then there’s the question of accurate foot tracking of course, but just like hand tracking, it’s really not that essential.)

          • Tommy

            I got ya. Never looked at it as surroundings tracking.

    • Ryan McClelland

      Seems like this should work!

  • rfanck

    What a surprise !

  • Cragheart

    The fix could be something you strap on your legs, like controllers you hold in hands. But PC VR with outside-in tracking is better anyway. We shouldn’t go in a mobile direction, but in direction with wireless connection to PCs. PCMR!

  • Tabp

    Reminder that the mainstream form of full body tracking in VR today uses inside out tracking via Valve’s lighthouse system, which is inside out and doesn’t rely on external cameras, since the lighthouses only generate light. You can even combine lighthouses and Vive or Tundra trackers with a Quest to get full body tracking using two different inside out systems at once.

    That puts Facebook in the awkward position of having to try to resurrect outside-in from the grave it was placed in when the Rift CV1 went out of production, only to end up with a body tracking system that would have disadvantages compared to an established competitor that can already be combined with Quest. Since Facebook isn’t willing to partner with Valve, it doesn’t make any business sense for them to try to reinvent the wheel unless they’re really committed to doing it properly at a very cheap price, which they’re not.

  • Ray Price

    Eventually we will have tiny inside-out independent trackers that we can easily attach to our bodies. Still a couple of years away though, I think. We don’t even have any TRUE inside-out controllers yet.