Leap Motion’s New 180-degree Hand-tracking Comes to Qualcomm’s Latest VRDK Headset

28

Qualcomm has debuted an updated version of their VR Headset Reference Design now with Leap Motion’s new 180-degree hand-tracking to bring gesture control to mobile VR headsets. The new headset and Leap Motion tracking module was shown off during last week’s GDC 2017.

Qualcomm’s VR Headset Reference Design has been upgraded to the company’s new Snapdragon 835 mobile platform. The purpose of the headset, which the company calls the VRDK (Virtual Reality Development Kit), is to act as a foundation for Qualcomm’s device partners to make their own VR headsets based on Qualcomm’s mobile computing hardware.

The VRDK is an impressive mobile headset itself, featuring a 2560×1440 AMOLED display and inbuilt inside-out positional tracking derived from internal sensors and a pair of 1280×800 front-facing cameras. We tried out the positional tracking earlier this year and found it to be quite performant.

qualcomm vrdk leap motion hand tracking (1)And now, Qualcomm’s latest VRDK brings hand-tracking into the mix thanks to Leap Motion. Leap Motion has been working on hand-tracking technology since 2010, and in recent years has pivoted their focus toward use as an input technology for VR. And while the company’s initial hand tracking device—originally built for use as a desktop peripheral—has seen some use in VR by strapping the device to the front of a headset, the limited field of view meant that users had to told their hands up in front of their face for the device to be able to track their hands in VR. Ultimately VR controllers have thus far become the defacto standard for motion input on tethered VR headsets.

But when it comes to mobile VR, where the goal is to have a single, self-contained unit that doesn’t rely on external tracking sensors or beacons, Leap Motion may have found a perfect fit; hand-tracking is more immersive than the limited rotation-only controllers that we see with Daydream and others (like the newly announced Gear VR). Having the tracking be totally on-board also means one less piece of equipment to tote around, helping to keep mobile VR portable and easy to use.

SEE ALSO
Samsung Announces New Gear VR with Touch Sensing Controller Included

Leap Motion identified this sweet spot a while back and has been teasing a new mobile solution that would address the field-of-view limitation that came from strapping the company’s pre-VR device onto VR headsets. The company formally announced the mobile made-for-VR module in late 2016, and now we’re seeing the first glimpses of integration into Qualcomm’s newest VRDK, which I got to try out at GDC 2017 last week.

qualcomm vrdk leap motion hand tracking (2)Though the new Leap Motion mobile module is technically still an attachment to Qualcomm’s VRDK, Leap Motion says that it will be directly integrated into mobile VR headsets built on the VRDK that opt for the hand-tracking tech.

qualcomm vrdk leap motion hand tracking (6)The new mobile module as seen at GDC 2017 hugs closely to the Snapdragon 835 VRDK and was clearly made to fit the device specifically. With two wide-angle lenses, Leap Motion says the module provides a 180 degree field of view for hand-tracking. Indeed, I could feel a significnat difference between the new module and the old one. With the headset on and my hands out in front of me, I could grab objects and let them out of my own field of view through the headset, and when I looked down I could see that I was still holding the object.

The increased tracking field of view is bolstered by smart tweaks to the hand-tracking software; such that if I was holding an object and then turned my head (causing the object to truly leave the tracking module’s field of view) the software would remember that I was holding that object (and in which hand) once it came back into view, and often identify my hand holding the object before it came back into the headset’s own field of view, making a big improvement from the compelling-but-frustrating experience of the original desktop module.

leap-motion-mobile-hand-trackingThe mobile module doesn’t just have a wider field of view, it’s also built for high power efficiency so that it makes sense to add to mobile VR headsets. Leap Motion says that the module runs at 10x the speed of the original device at significantly lower power. It’s also very tiny.

Continue Reading on Page 2 >>

1
2

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Sch@dows

    Leap motion does seem to work very well, but i’m still skeptical about the concept. Because when you’re not looking at your arms/hands, the game loose track of them, meaning you can’t take a look on you side and keep shooting, or turning the steering wheel, or even hold a virtual object.

    • NooYawker

      Not to mention feedback is an important part being immersed in VR.

      • Sch@dows

        Of course, but I was comparing (implicitly) the tech to the vivetracker glove, and any other hand tracking tech which also don’t offer any kind of feedback per se.

        From what I’ve seen, Leap Motion still works quite well even if your holding an accessories to get the feedback you need.

        • Andrew Jakobs

          And let’s not forget, the LeapMotion is much MUCH cheaper than any of the gloves or handtrackers at the moment (bought mine for not even 30 euro’s including taxes and postage).

          • Sponge Bob

            cause its obsolete – they are dumping inventories for previous product generation

          • Andrew Jakobs

            I bought mine about 2 years ago through amazon.

    • Jerald Doerr

      You took the words right out of my head…

    • Idea: arm-mounted leap motion cameras, themselves tracked by lighthouse. Capture the relative motion of the fingers and apply the arm transform to relocate it in the absolute world.

      Finger tracking and controllers use-cases don’t completely overlap. The user interaction they unlock are different. Being able to make a mark by directly squeezing space between your fingers for example. When really accurate it will be much more natural and intuitive.

      • Sponge Bob

        it will never be “really accurate”

        some dudes have ugly fat fingers

        others are missing some…

      • awilko

        Yes this, been waiting for this, but with mobile sixsense style non line of sight tracking rather than lighthouse which locks you to a room / PC. Shame about sixense though.

        edit: also, 2 cameras, one on either side of the wrist would solve most leap motion occlusion problems, eg, telling the difference between a closed fist and pointing away from camera.

    • OgreTactics

      That’s not entirely true. Hand-tracking, in terms of software and interaction is a bit more complex as well as flexible than we think, and that’s why Leap is doing a great job: first it has a 180° FOV which means that even when your hands are not in your VR FOV it’s still tracking it, but most importantly it doesn’t NEED to track it 1:1 or all the time. If you’re steering a wheel, even looking behind you, there’s a chance that part of your hand on the wheel is still in the Leap tracking. Same for shooting on the side, although shooting without looking doesn’t make sense. And if you are holding something, as long as your not contortioning yourself so that the object is behind you, it should track it and most importantly react as soon as you want to drop it or launch it.

      So the truth is that while 180° FOV hand-tracking is great, the FOV should be as wide (~225° like the ocular-FOV or slightly more) in order to track the hands in it’s widest range of regular actions. Also, in order to move, unless you have a castle or an Omnix-like threadmill, chance you’ll always have a small remote controller with you (like the GearVR tug or joycon) to accompany hand-tracking.

      • Sch@dows

        Indeed, I might have missed the 180° part of the article, which is a big improvement from the previous Leap Motion models, and could help with most cases (but not all unfortunately).

        As for the shooting without looking, it is something quite frequent. I’m not talking of doing that over long period of time, but shooting enemies while taking a look at what could lurk around you before going back to look at your target is pretty common.
        For example, I use it quite frequently on Space Pirate Trainer, going as far as shooting/finishing a drone on my right side, while peaking at my left side (or even above me) because a sound is alerting me another drone is preparing to attack.
        Same goes for driving games (but in my case I use a “real” gaming steering wheel meaning I don’t need to look at anything for inputting directions), where you look quite frequently to your sides during dogfights.

        In fact, turning you head 90° from where your arms are pointing will be enough to loose tracking. That is surely not normal situations, but it can still happen a lot.

        If the tech allows for 180° tracking, it could be helpful to have one on each side of the helmet to prevent blindspots.

        • Andrew Jakobs

          previous modeL, there is only one model.

    • Sponge Bob

      dud

      who cares about your gaming experience ?

      VR is for (future) productivity apps and those will need desktop mouse replacement, not stupid picture of all your 10 fingers (assuming you have all 10)

      • steve

        leap motion, myo are all cool things that could be used to make better vr experience

        • Sponge Bob

          cool – yes

          practical ? maybe

          myo is a joke – needs recalibration every time you move it

  • OgreTactics

    Leap Motion is the only company, which I now position as being the first modern virtual company, doing their job.

    No vaporware, no bullshit non-iteration, only advancement, streamlining and now integration into mobile.

    And it seems Qualcomm is the only other company doing it’s job, similar to what Intel did for CPUs until they weren’t offboard with their shit iGPU, the Snapdragon 835 is not only damn impressive for a mobile CPU but it’s also streamlining the integration of all the advanced software technology and hardware capabilities, that money-scrapping hardware/mobile manufacturers will unfortunately not use.

    Wireless, Inside-out/IMU motion tracking and hand-tracking are not VR functionalities or upgrades; VR simply doesn’t make one sense without them and a few other things that are not yet developed or integrated like Wireless tethering, see-through AR/environment-tracking and social eye/expression/voice tracking. Oh and also the design, I thought we we were through with the shit fixed strap design.

    • NooYawker

      That has yet to be seen. It’s easy to make believe your being innovative when you have no product. Leap Motion is mostly vaporware and a lot of bullshit.

      • Sponge Bob

        its not vaporware

        but it’s overhyped because this mode if interaction is limited

        by looking at your 10 fingers from OUTSIDE you can’t have sub-sub-mm precision required by some future apps (I would hate to have a VR-assisted surgery performed with leapmotion type of tech)

      • spo8

        I mean, if Qualcomm decided to integrate it into their reference headset, I’d guess that it at least functions.

      • Vampire Lestat

        You are probably thinking about Magic Leap. Leap Motion is not vaporware, their product just isn’t all that great yet/still. However, it has definitely gotten better at each iteration, so they are on the right track.

        • NooYawker

          You’re absolutely right. I got the two companies mixed up.

    • Sponge Bob

      BS

      For pure VR inside-out head tracking is not needed at all
      Do you walk from one room to another in your HMD ?
      AR on the go is different thing but that’s many years away

  • Andrew Jakobs

    The old (but still current) LeapMotion hardware is already a pretty solid experience. And with one performing 10 times better and a wider FOV it should be really good for a lot of jobs..

    • Wildtz0r

      I strongly disagree. The current hardware/software is too inaccurate to be considered usable.

  • Sponge Bob

    You can’t pinch your finger together with this – it feels completely fake because of “software trigger” tolerance threshhold they have to put in
    No real physical touch can be detected with optical tech. Period. (unless viewing angle is perfect which never happens) and no haptic feedback to the user is possible
    Good for some kids games but will not replace hand-held controllers (not talking about touch or vive’s – those are ugly monstrosities)

    And BTW, can Qualcom’s inside-out tracking which you hype so much replace solid outside-in tracking with sub-mm precision and a range of say >5 m in ANY room (white non-descriptive walls etc etc etc) ???

  • Sponge Bob

    Touchscreen DID NOT replace mouse-based interaction for productivity apps like Photoshop or Autocad

    Likewise, optical finger/hand tracking WILL NOT replace hand-held controllers for future VR productivity apps (3d “Photoshop”)

    Those future controllers will look nothing like today’s monstrosities (Touch of Vive)

    • josh

      Actually it replaced it in a few big areas. Most digital artists use a touch pad surface for photoshop and 3d modeling programs like 3dsMax and Maya.