In order to keep the visual system from thinking something is wrong, tracking has to be super-accurate.
On the order of a millimeter at 2 meters distance from the sensor.
There is currently no consumer-priced system that comes close to the required tracking accuracy and reliability across a wide enough range of motion.
So how close is VR tracking right now to what we really want?
Let’s look at what the tracking in the first Rift development kit can and can’t do.
The Rift uses an inertial measurement unit, or IMU, which contains a gyroscope and accelerometer.
IMU-based tracking is inexpensive and lightweight, which is good.
However, it also drifts because there’s no absolute positioning, and it doesn’t support translation – that is, it doesn’t provide accurate reporting of head movement from side to side, up and down, and forward and back, and that’s a significant lack.
Here we see translation in action. As the head moves from side to side, the virtual view changes accordingly, with nearer objects shifting by greater distances than farther objects. This produces parallax, one of the key depth cues and an important part of making virtual scenes seem real.
In the case of the Rift, translation has little effect on the virtual scene. I say “little” because there is a head and neck model that attempts to reproduce the translation of your head as it rotates, but it has no way to know about translation resulting from any other head movement.
For those of you clever enough to notice that my illustration of translation actually features a Rift, I admit that the translation in these screenshots didn’t come from the head movement – it was simulated by strafing from the keyboard. But that is how translation would look if the Rift did support it.
IMU tracking works for games that don’t require anything but head rotation – FPSes, for example.
But even in FPSes, the lack of translation means you can’t peek around corners or duck down.
In general, the lack of parallax makes virtual images seem less real.
Also, drift means that IMU-based VR can’t stay stable with respect to the real world, and that rules out games like board games that need to stay in one place.
This is definitely a long way from where we really want to be, although bear in mind that this is only the first development kit, and Oculus continues to work on tracking.