Apple is going all-in with hand-tracking for its upcoming Vision Pro, but can hand-tracking really replace proper VR controllers? Meta, Apple’s main competitor in the space, isn’t going so far as to make that particular claim, however the company says Quest’s recent hand-tracking improvements are now “almost as responsive as controllers” thanks to its recent v56 software update.

First announced in late July, the company’s Hands 2.2 tracking improvements are introducing what Meta says is better hand responsiveness and a few new experimental features that we’ll probably see in Quest 3.

Now rolling out to Quest headsets, Meta says users should notice hand-tracking latency reduced “up to 40%” in regular use, and “up to 75%” during fast movement. Meta says those dramatic gains in fast movement latency are thanks to the introduction of a new Fast Motion Mode (FMM) for more frenetic games, like fitness and rhythm games that require you to punch incoming objects.

Here’s a look at controllers relative to the new Hands 2.2 release:

v56 is also rolling out to Quest Pro, which includes two new experimental features: simultaneous hands and controllers (Multimodal) tracking, and controller-driven hand pose (Capsense Hands).

Apple Vision Pro Will Soon Let You Convert 2D Photos to 3D & Share Them Live via SharePlay

Multimodal tracking is said to unlock a number of mixed input style, including Hand+controller gameplay, instant transitions between hands and controllers, and improved social presence when using one controller. It’s only available on Quest Pro for experimentation, although Meta plans to add support for additional devices later.

On the other hand, Capsense Hands lets developers show a natural hand model visualization on top of, or instead of, a user’s controller. Check out both in action in the video below:

“Hand Tracking gives your app’s users the ability to engage with their VR or mixed reality environment in a way that is natural and intuitive,” Meta says in a developer blogpost. “These interactions deepen the immersive experience and help people feel more connected to what’s going on around them in-headset. Hands can also provide a faster on ramp for users new to VR. By integrating Hand Tracking in your existing apps, you can give your users more flexibility to tailor their experience and find what works best for them—and thanks to Hands 2.2, you can feel confident knowing your app’s users will enjoy a great experience regardless of whether they play with hands or controllers.”

Meta says we should hear more about its hand-tracking upgrades in the near future, as the company is hosting its annual Meta Connect developer conference on September 27th, which ought to include an info dump (and likely release date) for its upcoming Quest 3 headset, which is bringing much of the functionality of Quest Pro to the consumer price point of $500.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • ViRGiN


  • Andrey

    What I am really waiting for – when tracking speed and accuracy will finally reach the needed level – is an ability to use multimodality in shooters so your right hand (if you are a right-hand person of course) has a controller+stock/handguard attached to it and your left hand is just your hand, so you could very naturally grip the handguard when you need to without all those magnets and other crap that usually is in the way when you use a controller. Of course there will be some difficulties on the way – left stick won’t be accessible, so player will need to change movement control to a right stick and get used to it (or use a VR treadmill instead – ahh, wish there was an accesible one without Pico forcefully included…) + rotate in real world instead of using a stick for it. Not to mention tracking fingers properly when gripping something. And, of course, devs will need to make this possible on the software side (like making all the logic for gripping a handguard/magazines for a tracked hand with this API, etc.), but, imo, it’s not impossible. WIth something like that I would play VR shooters so much more and in much more enjoyable way than it is right now. Those old gunstocks were a thing when original Rift and Vive just released and there wasn’t any alternative, but now, when so many headsets have hand-tracking capabilities out-of-the-box, it’s a quality of life change that just should happen sooner or later.
    Hope one day we will see something like it implemented in Pavlov/Contractors/etc. or any new VR shooter.

  • Gonzax

    The problem is haptics. I don’t see any ‘normal’ game ever using only hand-tracking, at least not in a way that I would like to play those games but it can be great for other stuff like typing or certain casual games.

    • A big problem I see with it is there’s no longer a solid barrier between natural movement and intentional input. You reach up to scratch your chin, oops misread that as a click, just closed your program accidentally.

    • shadow9d9

      I’ve always found “haptics” to be garbage and annoying. Great for marketing though.People will gleefully parrot the marketing terms word for word, even when it adds nothing to the experience.

      So yeah, you want to carry and hold controllers just to feel some rumble and resistance vs just seemlessly interacting with the environment. Brilliant.

      • psuedonymous

        Haptics is not just buzzing motors. Haptics includes the tactile response of a button (the ability to place your finger atop a button, feel it is in the right post but not actuated, then feel the button actuate, and thus the ability to distinguish between a buttonpress that is registered but has a null effect, a finger motion that may or may not have been registered), the proprioceptive response of a thumb as it deflects an analog stick, etc.

      • Arno van Wingerde

        Hm: have you tried running your hand in the water at the start of Horizon: call of the mountain? That might change your opinion. I also think that turning a key into a lock feels better with some haptics. Otherwise, you can feel a bit like a ghost.
        To each his own, but for me haptics are the third most important part of VR, after view and sound. I wouldn’t mind if fans in the headset could simulate wind …

        • silvaring

          Basic Haptics is important for many VR game genres, but for simple UI interfacing, I don’t imagine you need anything more than subtle, low vibration feedback though.

    • Jeremiah Tothenations

      I agree, I could see them releasing haptic rings for your fingers eventually.

  • Has anyone else noticed that in v56 if you’re tethered to a computer using Link and you put down your controllers mid session, you can use hand tracking to go back to the Quest menu and start a recording? Real neat little addition, don’t know if that was possible before!

  • Nothing to see here

    I love the hand tracking on the Quest 2. The only major issue is that when it can’t see one of your hands, it starts to imagine it instead. When watching VR videos, I like to keep my arms relaxed at my sides. After a few seconds a phantom limb appears sticking out spastically towards the screen. This gets pretty annoying and there seems to be no way to avoid it. Even if you keep your hands in sight but motionless, they don’t go away after a while. The only way to fix this is to turn off hand tracking entirely.

  • Octogod

    I never thought hand tracking would feel good, but I often find myself using it in less frantic games or apps. It’s really magical.

  • Ardra Diva

    Unfortunately the games and apps still need the controllers, but in your home space, hand controls work really well. Surprisingly well.