Leap Motion’s New 180-degree Hand-tracking Comes to Qualcomm’s Latest VRDK Headset

28

The mobile hand-tracking module is running the ‘Orion’ branch of Leap Motion’s hand-tracking engine, which brought major improvements to the hand-tracking performance when used for VR.

In the Qualcomm VRDK I got to try the familiar ‘Blocks’ demo (above), which allowed me to use my hands to spawn shapes of varying sizes, which could be stacked, thrown or smacked. This time however, the demo was paired with the inside-out tracking of the Qualcomm VRDK, allowing me to physically walk around the virtual space to grab out-of-reach blocks. I could see my hands tracked well inside the headset, and in addition to being able to spawn shapes, I could also flip my hand palm-up to make a floating interface appear attached to that hand, and I could use the pointer finger on my other hand to tap various icons that impacted the experience.

qualcomm vrdk leap motion hand tracking (8)I was really impressed with the intuitiveness and functionality of this hand-based interface; it felt like an incredibly easy and natural way to use my own two hands to control my VR experience, likely because it kept the interface mounted to me, and used one of the most simple gestures known to humankind—the point/poke—to initiate actions. It felt very good and I’d love to see the idea adopted more widely, especially among mobile VR headsets which today have rather clunky interfaces.

This type of ‘hand-mounted interface’ isn’t entirely new—we’ve seen similar things in VR experiences that use VR controllers—but doing it without any controller in my hands was hands-down the more natural feel compared to pointing a laser pointer, or even to pointing your finger via Touch’s binary finger-sensing implementation (which does allow you a ‘point’ gesture).

SEE ALSO
Meta Reportedly Rejected Google Partnership to Bring Android XR to Quest

Impressive tech, but still there’s still a number of limitations that this sort of hand-tracking tech will be hard pressed to solve. For one, physical buttons and sticks are still important input methods; I still get a more natural and immersive feeling by using a trigger to grab an object in VR than by pinching my fingers together into a ‘fake’ grabbing gesture. The increased field of view certainly helps, but there’s still times where your hand will leave the hand-tracked field of view.

Despite these challenges, when it comes to mobile VR, Leap Motion’s new mobile tracking module is a compelling addition, and, if the price is right, a seeming no-brainer. It has the potential to make the mobile VR experience much closer to the that of high-end tethered VR headsets, thanks to the ability to naturally reach into the virtual world in a way that’s much more immersive than the basic rotational controllers of today’s mobile VR headsets. Not to mention the usability factor—if mobile VR headsets are aimed at more casual users, the ability to control an interface by reaching out and touching it with your finger is something that everyone with a smartphone already knows how to do. That means mobile VR headsets equipped with hand-tracking are likely to have a much reduced learning curve compared to basic rotational controllers and to the side-head-mounted trackpad approach that we see in Gear VR and other headsets.

Qualcomm says the first VR headsets built on the VRDK are due out in 2017, but whether they will incorporate the Leap Motion mobile hand-tracking tech isn’t clear just yet.

SEE ALSO
Apple Reportedly Expands Vision Pro Production Capacity Following Successful US Launch

Disclosure: Qualcomm and Road to VR co-hosted a networking event during GDC 2017.

1
2

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Sch@dows

    Leap motion does seem to work very well, but i’m still skeptical about the concept. Because when you’re not looking at your arms/hands, the game loose track of them, meaning you can’t take a look on you side and keep shooting, or turning the steering wheel, or even hold a virtual object.

    • NooYawker

      Not to mention feedback is an important part being immersed in VR.

      • Sch@dows

        Of course, but I was comparing (implicitly) the tech to the vivetracker glove, and any other hand tracking tech which also don’t offer any kind of feedback per se.

        From what I’ve seen, Leap Motion still works quite well even if your holding an accessories to get the feedback you need.

        • Andrew Jakobs

          And let’s not forget, the LeapMotion is much MUCH cheaper than any of the gloves or handtrackers at the moment (bought mine for not even 30 euro’s including taxes and postage).

          • Sponge Bob

            cause its obsolete – they are dumping inventories for previous product generation

          • Andrew Jakobs

            I bought mine about 2 years ago through amazon.

    • Jerald Doerr

      You took the words right out of my head…

    • Idea: arm-mounted leap motion cameras, themselves tracked by lighthouse. Capture the relative motion of the fingers and apply the arm transform to relocate it in the absolute world.

      Finger tracking and controllers use-cases don’t completely overlap. The user interaction they unlock are different. Being able to make a mark by directly squeezing space between your fingers for example. When really accurate it will be much more natural and intuitive.

      • Sponge Bob

        it will never be “really accurate”

        some dudes have ugly fat fingers

        others are missing some…

      • awilko

        Yes this, been waiting for this, but with mobile sixsense style non line of sight tracking rather than lighthouse which locks you to a room / PC. Shame about sixense though.

        edit: also, 2 cameras, one on either side of the wrist would solve most leap motion occlusion problems, eg, telling the difference between a closed fist and pointing away from camera.

    • OgreTactics

      That’s not entirely true. Hand-tracking, in terms of software and interaction is a bit more complex as well as flexible than we think, and that’s why Leap is doing a great job: first it has a 180° FOV which means that even when your hands are not in your VR FOV it’s still tracking it, but most importantly it doesn’t NEED to track it 1:1 or all the time. If you’re steering a wheel, even looking behind you, there’s a chance that part of your hand on the wheel is still in the Leap tracking. Same for shooting on the side, although shooting without looking doesn’t make sense. And if you are holding something, as long as your not contortioning yourself so that the object is behind you, it should track it and most importantly react as soon as you want to drop it or launch it.

      So the truth is that while 180° FOV hand-tracking is great, the FOV should be as wide (~225° like the ocular-FOV or slightly more) in order to track the hands in it’s widest range of regular actions. Also, in order to move, unless you have a castle or an Omnix-like threadmill, chance you’ll always have a small remote controller with you (like the GearVR tug or joycon) to accompany hand-tracking.

      • Sch@dows

        Indeed, I might have missed the 180° part of the article, which is a big improvement from the previous Leap Motion models, and could help with most cases (but not all unfortunately).

        As for the shooting without looking, it is something quite frequent. I’m not talking of doing that over long period of time, but shooting enemies while taking a look at what could lurk around you before going back to look at your target is pretty common.
        For example, I use it quite frequently on Space Pirate Trainer, going as far as shooting/finishing a drone on my right side, while peaking at my left side (or even above me) because a sound is alerting me another drone is preparing to attack.
        Same goes for driving games (but in my case I use a “real” gaming steering wheel meaning I don’t need to look at anything for inputting directions), where you look quite frequently to your sides during dogfights.

        In fact, turning you head 90° from where your arms are pointing will be enough to loose tracking. That is surely not normal situations, but it can still happen a lot.

        If the tech allows for 180° tracking, it could be helpful to have one on each side of the helmet to prevent blindspots.

        • Andrew Jakobs

          previous modeL, there is only one model.

    • Sponge Bob

      dud

      who cares about your gaming experience ?

      VR is for (future) productivity apps and those will need desktop mouse replacement, not stupid picture of all your 10 fingers (assuming you have all 10)

      • steve

        leap motion, myo are all cool things that could be used to make better vr experience

        • Sponge Bob

          cool – yes

          practical ? maybe

          myo is a joke – needs recalibration every time you move it

  • OgreTactics

    Leap Motion is the only company, which I now position as being the first modern virtual company, doing their job.

    No vaporware, no bullshit non-iteration, only advancement, streamlining and now integration into mobile.

    And it seems Qualcomm is the only other company doing it’s job, similar to what Intel did for CPUs until they weren’t offboard with their shit iGPU, the Snapdragon 835 is not only damn impressive for a mobile CPU but it’s also streamlining the integration of all the advanced software technology and hardware capabilities, that money-scrapping hardware/mobile manufacturers will unfortunately not use.

    Wireless, Inside-out/IMU motion tracking and hand-tracking are not VR functionalities or upgrades; VR simply doesn’t make one sense without them and a few other things that are not yet developed or integrated like Wireless tethering, see-through AR/environment-tracking and social eye/expression/voice tracking. Oh and also the design, I thought we we were through with the shit fixed strap design.

    • NooYawker

      That has yet to be seen. It’s easy to make believe your being innovative when you have no product. Leap Motion is mostly vaporware and a lot of bullshit.

      • Sponge Bob

        its not vaporware

        but it’s overhyped because this mode if interaction is limited

        by looking at your 10 fingers from OUTSIDE you can’t have sub-sub-mm precision required by some future apps (I would hate to have a VR-assisted surgery performed with leapmotion type of tech)

      • spo8

        I mean, if Qualcomm decided to integrate it into their reference headset, I’d guess that it at least functions.

      • Vampire Lestat

        You are probably thinking about Magic Leap. Leap Motion is not vaporware, their product just isn’t all that great yet/still. However, it has definitely gotten better at each iteration, so they are on the right track.

        • NooYawker

          You’re absolutely right. I got the two companies mixed up.

    • Sponge Bob

      BS

      For pure VR inside-out head tracking is not needed at all
      Do you walk from one room to another in your HMD ?
      AR on the go is different thing but that’s many years away

  • Andrew Jakobs

    The old (but still current) LeapMotion hardware is already a pretty solid experience. And with one performing 10 times better and a wider FOV it should be really good for a lot of jobs..

    • Wildtz0r

      I strongly disagree. The current hardware/software is too inaccurate to be considered usable.

  • Sponge Bob

    You can’t pinch your finger together with this – it feels completely fake because of “software trigger” tolerance threshhold they have to put in
    No real physical touch can be detected with optical tech. Period. (unless viewing angle is perfect which never happens) and no haptic feedback to the user is possible
    Good for some kids games but will not replace hand-held controllers (not talking about touch or vive’s – those are ugly monstrosities)

    And BTW, can Qualcom’s inside-out tracking which you hype so much replace solid outside-in tracking with sub-mm precision and a range of say >5 m in ANY room (white non-descriptive walls etc etc etc) ???

  • Sponge Bob

    Touchscreen DID NOT replace mouse-based interaction for productivity apps like Photoshop or Autocad

    Likewise, optical finger/hand tracking WILL NOT replace hand-held controllers for future VR productivity apps (3d “Photoshop”)

    Those future controllers will look nothing like today’s monstrosities (Touch of Vive)

    • josh

      Actually it replaced it in a few big areas. Most digital artists use a touch pad surface for photoshop and 3d modeling programs like 3dsMax and Maya.