There’s still a lot of unanswered questions surrounding HTC’s Steam VR Headset the Vive, not least of which how exactly the tracking solution works. There is one theory that has emerged which does offer a plausible solution.

Yesterday, in the chaos of a (mostly) surprise announcement from HTC – that they’d be manufacturing for Valve the flagship, reference VR headset to delivery Steam VR’s virtual reality experiences – we posted a breaking story which stated the new device uses an inside-out tracking solution, based on the sensor array visible on an early, official render of the device. We later retracted this statement as there was no hard evidence at the time to support this claim.

After some thought since then and a lot of community discussion, we think the statement has some merit. And although this is purely speculation right now (we’ll find out soon enough how Valve’s vision for VR works at GDC this week), we also figured the theories were interesting enough to explore.

The Oculus Rift DK2, which debuted at GDC 2014, in a bid to resolve issues such as simulation sickness – caused by a disconnect between the visuals a VR headset displayed and your physical movements – introduced its IR ‘outside in’ camera-based positional tracking system. The camera picks up IR LEDS on the face and sides of the headset, which pulsate at determined frequencies from which absolute orientation and translational position can be determined, as long as the headset remained in the field of view of the camera – outside of which the headset defaults to on board sensors. This is called ‘outside-in’ tracking, as the observer device is based off-headset, looking at the object which is being tracked.

SEE ALSO
Sequel to Classic Adventure Game 'MYST' is Coming to VR This Year
oculus-rift-dk2-developer-kit-2-specs
The Oculus Rift DK2 and IR Tracking Camera

Back in January 2014, at Steam Dev Days, a couple of months before the DK2 made its debut, people were being wowed by a prototype headset, developed by Valve’s R&D lab, which used dual LCD panels and a positional tracking system which used printed, fiducial markers (think QR Codes) places on the walls, ceilings and floor of a special demo room. Headset mounted cameras used these markers to determine the user’s absolute position in 3D space by tracking the positions and orientation of these codes. This is an ‘inside-out’ tracking system, so called because the observer devices are mounted on the tracked device, determining it’s position using information from the world around it.

Valve's VR demo room at Steam Dev Days
Valve’s VR demo room at Steam Dev Days

So it’s not a huge stretch to think that Valve would evolve that system for use in their new, commercial headsets. Clearly though, no one wants to have to print off 60 markers to stick all over their house, so how can Valve offer the same level of tracking fidelity in a package suitable for use by the average Joe?

The answer may be in the information we already have. The initial specifications, as displayed on the new HTV Vive website states:

A gyrosensor, accelerometer, and laser position sensor combine to precisely track the rotation of your head on both axes to an accuracy of 1/10th of a degree, allowing you to look around the virtual environment naturally.

..and perhaps more tellingly:

Couple the headset with a pair of Steam VR base stations to track your physical location (in spaces up to 15 feet by 15 feet) – get up and walk around insIde the virtual world!

The theory, as explored on this /r/oculus thread, is that those base stations act as projectors, throwing IR versions of the fiducial markers seen in the Steam Dev Days demo room, out onto all surfaces of the user’s room. The sensors on the Vive then pick up the markers, tracking the user in 3D space in a similar manner.

SEE ALSO
The Sequel to One of Quest's Best-rated Games is Now Available, Trailer Here

Romantic-Star-Projector-1-Design-Crush

The technology may bear some resemblance to this project, which used a central projector which threw out IR lights into the room. This forum thread has Palmer Luckey, inventor of the Oculus Rift, stating he was working on a project similar to this back in January 2014. Given Oculus’ close relationship since the launch of the Rift’s Kickstarter, it’s not much of a leap these ideas were being shared. At the very least, it means the idea probably has some technical merit.

We’ll find out more very soon of course. GDC 2015 is about to kick off, and Valve is due to demonstrate it’s new hardware to delegates at the show, including our very own Road to VR reporters on the ground. We’ll of course report back once we have more detailed information on the HTC Vive and Steam VR.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Based in the UK, Paul has been immersed in interactive entertainment for the best part of 27 years and has followed advances in gaming with a passionate fervour. His obsession with graphical fidelity over the years has had him branded a ‘graphics whore’ (which he views as the highest compliment) more than once and he holds a particular candle for the dream of the ultimate immersive gaming experience. Having followed and been disappointed by the original VR explosion of the 90s, he then founded RiftVR.com to follow the new and exciting prospect of the rebirth of VR in products like the Oculus Rift. Paul joined forces with Ben to help build the new Road to VR in preparation for what he sees as VR’s coming of age over the next few years.
  • RedLeader

    I’m still a bit skeptical of how this will work. We’ve seen similar projects in the past try something akin to this, and they haven’t worked out very well. The one that comes most readily to mind is the MS Kinect. I know it’s a different technology from the cast IR lights displayed above, but the challenges are the same.

    The biggest hurdle with projecting these lights onto surfaces are obstructions by other objects. Assume for a moment that you did place a sensor on the floor, ala the second image. The user’s chair obstructs the lights from hitting the computer desk, so that whole area is invisible to the sensors.

    As the player moves about the room, the player themselves will constantly be blocking different IR lights. The two sensors working in tandem can figure this out, but as we’ve seen with laser and IR scans in the past, with much more precise, much, much more expensive equipment, there is still a lot of lost fidelity in the scans that have to be filled in or re-run at different positions to cover the gaps in the scans, and the scans themselves are slow.

    I also know that my home office doesn’t look much like the room above. There are two lamps, the desk is at an angle against the corner (for maximum leg room), there are two chairs (an office chair and a lounge chair), and there are other objects in the room. How will having a non-square or non-rectangular room affect this type of deployment?

    I have to imagine this is why Oculus skipped the “outside-in” approach on the DK2 and Crescent Bay designs – a depth camera with IR sensors “just works”, not as well as Valve’s VR room, but in a much more realistic consumer device-oriented way. The method proposed in the article seems likely to be more buggy than the current Rift implementation.

    I think it’s more likely that there is an IR camera ala the Rift and then there is also the use of the positioning lasers from the headset to measure the distance to objects, so that the user can walk around a room with the lasers measuring distance to walls (still doesn’t stop the user from running into chairs and tables- maybe that’s what the two stations help detect?) and avoid running into them.

    How will this play out in games? Will games have audible or visual cues when someone approaches an object IRL? Will games be designed around a common size square space? Will this not be used for games at all, but for interactive entertainment ala Oculus’ Crescent Bay demos?

  • zalo

    It all comes down to whether or not those are LEDs or photodiodes on the headset.

    It could be a combination of the suggested mechanism plus an outside in camera for correction if those are LEDs, or something else entirely…

  • Aaah, speculation, my brain does it all the time.

    I suspect the surface sensors to be light/photo sensors >_O I have no idea what they are for, but as they are positioned in indents they’re certainly not made to be seen from many direction, but to see in certain directions.

    The lack of lens does make me wonder just how this works, it should mean that the sensor does not have to focus incoming light, but that incoming light is already focused. This rhymes well with lasers being used in the base stations. My instant theory here is that perhaps the sensors will notice scanning laser lines, perhaps sweeping both vertically and horizontally, this way they might be able to pin-point position.

    One thing I wonder though, is if the bases are passive, no data connection to the headset just doing their thing while all the sensing is done by the HMD. If so they might emit a laser grid of sorts… hmm. Well, it’s a thought at least.

    I’m betting the same technique used for the headset is used in the controllers, slap a number of those sensors on the outside of a handle-like controller and you are good to go, angle them properly and it might suffice to have one Steam VR base at each opposite corner of a room to get good coverage.

    I am certain I will furnish my room to work with this system, haha. Right now I have placed my furniture in such a way I can get the DK2 camera up high so I’m already doing this :P

  • gerwindehaan

    @Andreas I was also wondering about the absence of lenses, indicating these could indeed be photodiode (2D) arrays instead of regular cams. Potentially, the two bases could each emit (IR) laser light through a passive encoded film and lens, resulting in a structured (IR) pattern that encodes location throughout the room, somewhat along the lines of the wonderful Lumitrack (http://chrisharrison.net/index.php/Research/Lumitrack — thanks to @anthony_steed for pointing that one out)

    • That is really interesting, kind of what I imagined but much denser. It cannot be too dense or the sensors might get confused >_O And I do wonder if they’ll use actually visible light or IR. IR would make more sense if other people are in the room, if not, does’t matter until you’re outside the headset I guess. Wouldn’t want laser light in my eyes. That said, how dangerous is IR laser for the eyes, dangerous at all? Depends on exposure? Should just generate heat eh?

      An option would be to alternate the patterns, but then they’d need some sort of sync between the headset/controllers and the bases, I think. Now, HTC still called the bases for Steam VR bases, but I wonder if they’re actually part of the kit they make. Very hmm indeed.

  • George

    Very interesting. This seems to be aiming for a higher price point than Oculus (2 screens, 70 “sensors”), so may not directly compete with them. I wouldn’t mind paying more if the resolution and FOV were a significant upgrade without some other major trade offs (like cheaper screens, not OLED or comparable).

    They also mentioned audio in their presentation, though I don’t see any headphones. I really like that Oculus is focusing on audio, and hope other VR companies don’t skimp on it.

    • RedLeader

      > “They also mentioned audio in their presentation, though I don’t see any headphones.”

      Well, according to BGR, the HMD has headphone jacks, so you can plug directly into the HMD instead of running two separate cables to the PC. Maybe that’s what your source was referring to?

  • George

    This is a link to the video I watched
    http://youtu.be/BWhM_oHr558?t=23m37s

    • RedLeader

      At work, so I don’t have the sound on (no headphones), so I will have to check it out later. Here’s the link I was talking about from BGR: http://bgr.com/2015/03/01/htc-vive-virtual-reality-headset/

      Their paragraph reads “A headphone jack sits on the side of the Developer Edition so that you can plug in your favorite earplugs or headphones. A gyrosensor, accelerometer, and laser position sensor are all contained within the device as well.”

      It’s possible that HTC/Valve are working on some integrated headphones ala the Crescent Bay demo, but like you, I haven’t seen any in the images. Plus, I have decent headphones; integrated headphones are not a big selling point for a lot of people – maybe they’ll stick w/ just the jacks?

      • Jacob Pederson

        Headphones are not a big deal to me for sure, but Oculus’s binaural audio support in the API is pretty cool. I’d take the freedom to walk around in a real-world room over binaural audio any old day though :)

  • WadeWatts

    Like any new non-shipping technology we need to look at all the details and how it relates to the creation of the software experience. I think the big take-a-way here is standing VR. There are many experiences that are better standing up. This announcement is sure to bring great happiness to all the creators of the VR treadmills as well. GDC is shaping up to be a seminal event for VR. I will be flying out tomorrow morning for what is sure to be an amazing show for VR developers. The media will be flooded with reports and critiques of what this device is capable of. At the end of the day the hardware is only as good as the software. Time to go down to the metal on my VR project! Woot!