Valve’s Index headset has had a passthrough view feature since launch, but the company has just released an experimental update which aims to make the view through the cameras more realistic by creating a synthetic image that more closely matches what your eyes should actually see.
A week ago, if you put on a Valve Index headset and double-tapped the menu button, you’d be greeted with a ‘passthrough’ view of your surroundings. Although this view is technically stereoscopic, it isn’t showing you exactly what you’d see if you weren’t wearing the headset, largely because the cameras on the headset are not in the exact same position as your eyeballs. In fact, the cameras are several inches away from where your eyes actually are, and they’re spaced much further apart.
How to Opt into SteamVR Beta
In your Steam games list, right-click on SteamVR > Properties > Select the ‘Betas’ tab
In the drop down list, select ‘SteamVR Beta Update’
Allow SteamVR to update
This week Valve released and experimental upgrade to Index’s passthrough view that’s designed to fix these problems and give you a ‘stereo-correct’ view. Valve calls it ‘Room View 3D’, and if you have an Index headset yourself you can check it out today by installing the SteamVR beta version 1.13, and then enabling ‘3D’ in SteamVR’s ‘Camera’ options. In this update, Valve also updated Index’s firmware to reduce the latency of the headset’s cameras.
As for Room View 3D our understanding is that, rather than simply showing you exactly what the cameras are seeing, the view from each camera is being compared to assess the depth of each part of the scene. The scene is then ‘distorted’ from the raw camera input to more closely match what a pair of cameras would see if they were in the exact same position as your eyes.
We tested out Room View 3D and could see the benefits of the stereo-correct view. When the system was working correctly, everything felt a bit more realistic, especially the size of your hand in front of the headset.
However, the system is highly unstable at this experimental stage; even just looking slowly around the environment would show depth artifacts which look like the parts of the scene are glitching. Occasionally when bringing my hand close to the headset it would be stretched and distorted across several different depths. I would guess that the low light sensitivity of Index’s cameras are a contributing factor.
Valve’s Room View 3D approach is similar to the ‘Passthrough+’ feature on Quest and Rift S, though Oculus’ solution is much more stable and usable at this point, even in similar lighting conditions. Valve’s “experimental” approach will ostensibly improve over time; the company notes that “this functionality is rapidly evolving, so we want to hear your thoughts and feedback.” Valve says it worked with Arcturus Industries and Occipital to create the underlying computer vision technique.
Ahead of the headset’s launch, Valve talked up the cameras on Index as being ideal for computer vision, but nearly a year after launch the company had done very little with them beyond the initial passthrough view—until now. Unfortunately the original passthrough view and the new Room View 3D continues to suffer from a long (several second) activation delay, making the feature much less useful for quick peaks outside of the headset. By contrast, activation of passthrough on Oculus’ headsets is nearly instant, making it more viable for quick looks to talk to someone or to make sure nothing has entered your playspace.
Valve says that users can “expect to see more experiments over the coming months,” and there’s hope that the work could eventually result in the passthrough view being used in SteamVR’s boundary setup process, as Oculus’ latest headsets have proven that this makes for a more fluid setup experience.