Performance & Limitations

I was impressed at the stability of the hand-tracking, which almost never had any immersion-breaking hand spasms that sometimes occur when computer-vision based hand-tracking systems aren’t quite sure what your hands are doing. I was also very impressed with the system’s ability to read some of the more challenging hand-poses, like when the back of my palm obscured my fingers with my arm held out in front of me. Even small movements of my knuckles peering over the top of my hand were reflected pretty well in virtual reality.

That said, the system is clearly tuned to aggressively hide any mistaken tracking, and does so by making your hands completely disappear when it’s uncertain where they are or what pose they’re making. I saw this happen often when I was testing the limits of the hand-tracking system; it especially didn’t like when my hands were touching or overlapping at all, and would quickly disappear them.

SEE ALSO
Hands-on: Dexmo Haptic Force-feedback Gloves are Compact and Wireless

In terms of tracking coverage, hands felt well tracked within Quest’s visible field of view, but I wasn’t able to get a sense for how much further it did or didn’t extend from there.

When it comes to performance and technical capability, hand-tracking on Oculus Quest felt immersive and entirely up to the task of a casual built-for-hand-tracking experience like Elixir, and surely for operating basic functions of the headset like navigating menus and launching content. Oculus will likely improve hand-tracking over time, and will hopefully be able to dial back the frequency of hand disappearances.

Less Friction Means Wider a Wider Appeal for Quest

For those who aren’t used to gamepads, understanding how to use a VR controller continues to be a major point of friction to using VR effectively. The ability to use the headset simply by pointing and touching with your hands could significantly broaden Quest’s appeal and be a major boon for use-cases where the precision and reliability of controllers isn’t necessary and where immersion isn’t the top priority, like this training experience which Oculus also demoed at Connect:

Casual users who want to watch media, browse the web, or join a virtual chat room with friends will be well served by the hand-tracking without bothering with controllers. Similarly, commercial and enterprise uses of VR will make good use of the reduced friction of hand-tracking in places like training, marketing experiences, design review, remote meetings, and much more.

Core Games Will Still Rely on Controllers

Image courtesy Oculus

That said, controllers are very likely to remain the dominant form of input for core gaming experiences because of their precision, reliability, and wealth of inputs. Hand-tracking will work well for casual experiences where users are poking, touching, or pinching, but for games where players will hold and interact with complex objects (like guns, bows, swords, grenades, levers, etc), controllers will continue to offer much more depth.

One reason is that, for hand-tracking, the headset only knows your intended input when your hands are clearly visible. This makes it difficult for the system to know if you’re choosing to continue to hold an object when your hand moves outside of tracking range. A controller, however, can continue to transmit that the ‘grab’ button is held no matter if the headset’s cameras can see it or not.

Another reason is that moving around inside VR environments often makes use of sticks, buttons, or triggers, and in core gaming scenarios, players expect this to be consistent and precise. Without a highly reliable input, moving around quickly and effectively would be challenging.

Similarly, many games have a high frequency of inputs (like pulling the trigger of a gun), and these must be 99.9% consistent to prevent frustration. In a game where you might trigger some input (be it buttons, sticks, or triggers) 500 times during a session, even 90% reliability would mean 50 failed inputs—put any controller in a core gamer’s hands with buttons that only work 90% of the time and you have a recipe for a broken TV screen!

Hand-tracking gestures must be very deliberate in order to be consistently recognized, and it’s been a persistent challenge for any computer-vision based hand-tracking technology so far to achieve 99.9% consistency for the sorts of binary ‘yes/no’ ‘on/off’ inputs which are used liberally in core gaming experiences.

SEE ALSO
Hands-on: HaptX's VR Glove is the Closest I've Come to Touching the Virtual World

These are just a few reasons why content built for hand-tracking necessitates different design choices than content built for controllers, and why hand-tracking is well suited for casual input scenarios while controllers will continue to see significant use for gaming.

Indeed, Oculus itself is not positioning hand-tracking as the next-gen input paradigm designed to replace controllers on Quest. At Connect a member of the team working on hand-tracking told me that they see it as an option for developers, but not a replacement for controllers.

Quest is due to get its hand-tracking update sometime in early 2020, and while there haven’t been any official announcements yet the company told us it’s considering bringing the feature to Rift S as well.

1
2

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • Really excited to try this out. It seems like it’ll be great for simple input situations like browsing the internet or watching Netflix.
    I also have to wonder how the LBE crowd feels about this feature. There are a number of experiences out there that use Leap Motion for hand tracking. Having it essentially baked into the headset with a much larger tracking volume seems like potentially a big win.

  • Pablo C

    If this is good enough to finger-track to a digital keyboard, it will be gamechanger. It could make the Quest a production tool.

  • Mateusz Pawluczuk

    “the headset only knows your intended input when your hands are clearly visible. This makes it difficult for the system to know if you’re choosing to continue to hold an object (…)”

    That’s where Ctrl-Labs brain reading wristband comes in ( ͡° ͜ʖ ͡°)

  • As someone told on Twitter, this is not comparable with Leap Motion yet. But Facebook has plenty of money to invest. And this is running on a mobile headset

  • kontis

    It’s extremely strange that this article doesn’t mention the most important formation: a comparison with Leap Motion.

    • Trip

      I suspect that it will be superior to Leap Motion due to the form factor if nothing else. We know Leap Motion has only two cameras that are very close together. It also appears to have a far inferior camera resolution when compared to Quest and Rift S. Most importantly, Leap motion has been around since before VR and it still has had very little integration since the adoption rate is so low. Every year or so I dig mine out and do battle with it for a couple days trying to get decent controller emulation out of it, then put it back away in disappointment. The problem is the lack of official support, not the device. A feature that every Quest (and hopefully Rift S) user has by default would start to give devs a reason to support this method of input. I’m also very much hoping Oculus will allow the hand tracking to do it’s best to emulate Touch controls across all apps. That would be a huge selling point.

  • Trip

    Hi Ben. Do you think that Oculus will give the option to have your hands emulate Touch controllers across all apps? I know it would be terrible in many applications so I wouldn’t be surprised if they don’t but I really wish they’d give us the tools and let us decide what we want to use it in. Might you be able to put in an inquiry with Oculus?

  • Maddy

    this would be cool in racing games ! Like dirt rally 2.0.
    i want to see my hands synchronized there :)