The first version of Varjo’s ‘bionic display’ enterprise VR headset is due out later this year (see our latest hands-on), and the company has announced plans to offer an add-on upgrade in 2019 which brings high-quality pass-through AR capabilities to the headset. Varjo showed off a tech demo of the headset’s add-on capabilities this week at AWE 2018.

Given Varjo’s positioning (exclusively for non-consumer sectors), the company is aiming for maximum quality visuals for their AR add-on. As shown in their current tech demo headset, it houses a pair of 4K cameras with lenses that match the headset’s own ~95 degree field of view, supporting stereoscopic pass-through video at 90Hz. Latency for the pass-through input is presently around 40ms, the company says, but they expect to be able to achieve single-digit latency before the add-on launches.

A look through their AR tech demo headset (which is identical to the current Varjo Beta headset, with the addition of the AR add-on) showed a fair bit of noise in the low lighting of the demo space; despite this, it was clear from a visual quality standpoint that Varjo has presented the best looking pass-through AR I’ve seen yet. Inside the headset I saw the view of the real room around me, while a full-sized model of a virtual motorcycle was projected into the space. Given the headset’s unique inset display, the center of my field of view revealed crisp details on the motorcycle that couldn’t be seen otherwise. Compared to optically transparent AR headsets, the field of view of both the real world and the virtual content was wide and immersive.

Photo by Road to VR

The brief demo I saw was purely visual for now, and lacked interaction. Varjo isn’t talking about advanced functions which one would hope to see with a pass-through AR system, like hand-tracking or SLAM; so while it’s clear that Varjo is aiming for high quality visuals in their AR add-on, it isn’t clear how much of the input/interaction pipeline they intend to own.

SEE ALSO
Hands-on: Varjo's Latest 'Beta' Prototype is a Promising Shortcut to Retina Resolution

Varjo released a video showcasing the upcoming AR pass-through add-on, though there’s an important caveat to note before viewing it: the company says that it was “shot entirely through Varjo’s [AR] prototype in real-time,” but this is somewhat misleading, as the video was not recorded through the headset’s lenses. Instead, it appears they recorded the input from the headset’s camera, and while it’s mostly representativie of the quality you’d see on the headset’s focus display, it doesn’t represent the quality drop off of the peripheral context display. More simply put: it’s a direct video feed instead of being shot through the lens of the headset, and the part you’re seeing represents the quality of the headset’s high resolution center display, not the lower resolution peripheral area.

Caveats aside, what they’re showing here is rendered in real-time, and the company says that their software is interpreting the scene’s real world lighting to allow the virtual components to cast convincing shadows onto the real ground, and for elements of the real scene to be cast in reflections on virtual objects (in the last shot you can see that the puddle reflections on the ground are actually virtual.

A render of what Varjo expects their AR add-on will look like at launch. | Photo by Road to VR

There’s pros and cons to doing pass-through AR instead of transparent AR. For one, the augmented parts of the world can look quite a bit more real because the pixels are drawn directly on top of the image of the real world, eliminating that semi-transparent ‘hologram’ look that you’ll find on transparent AR displays (caused by the fact that it’s extremely difficult to create pixels that can occlude 100% of the light from the outside world when dealing with a transparent display). It’s also much easier today to achieve an immersive, wide field of view by using pass-through AR rather than transparent AR.

SEE ALSO
Update: ZED Mini Turns Rift and Vive into an AR Headset From the Future

On the flip side, our eyes have excellent resolution and contrast ratio, which generally means we limit our dynamic range (the ability to see dark and light areas of a scene at the same time) and visual fidelity when using pass-through AR (not to mention the potential to introduce latency).

Update (6/3/2018): A prior version of this article stated that the pass-through video was only showing the cropped view that would be displayed on the headset’s central focus display. This has been corrected to indicate that the pass-through video shows the footage that would be displayed across both displays, but is shown at a quality level which matches the headset’s cameras without showing the pixel density drop off between the focus display and the peripheral context display.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • Hi Ben, thank you for the article!

    This one detail you unfortunately got a bit wrong: “and used a cropped portion of the frame that represents the part of the input footage that would be shown on the headset’s central focus display.”

    The video is from the wide view as you can probably tell from the wide FOV – it’s not from the cropped part. (Technically speaking it’s the image from the context display.)

    • benz145

      My apologies, I did mix that up, issuing a correction now.

  • Lucidfeuer

    Design and branding is always a great aspect on which to judge promises. And it seems they’re getting that right. Now I don’t know about the “real-time” video (which clearly uses some kind of video/filters or treatments), looks to good to be true, but I hope R2VR will be able to do a live review soon. The only thing I do not like are the bulky, misplaced cables.

  • Nate Vander Plas

    This makes me think that pass-through AR may actually become the standard over the “traditional” transparent display AR. To get more dynamic range (like HDR) all you would need is eye tracking that adjusts the exposure of the camera based on where you’re looking. Eventually (I hope) we’ll have light field displays inside a headset that can capture the world in front of us, add some stuff, and display it to our eyeballs in true 3D with no convergence/accommodation conflict.

    • Nate Vander Plas

      I guess one of the few upsides to transparent displays is that people could potentially see your eyes and still interact with you in the real world. Unfortunately, though, most of the “transparent” displays I’ve seen are pretty hard to see through from the outside. Maybe future pass-through headsets could have a second display on the outside that uses eye tracking data to animate a CGI version of your eyes. Haha!

    • Robert Smith

      Abslutely agree. Take an oculus half dome prototype with it’s accomodation ability, high FOV and res, slap some hi res fisheye cameras with depth mapping on the front + maybe a few area scanners = probably the best solution for true AR in the near mid term. Hololens and Magic Leap type tech just isn’t anywhere near where it needs to be.

      The milltary etc. applications are clear too – if the cameras are IR etc. sensitive then this becomes a superior nightvision setup, not to mention the ability to patch in HUD stuff, maps, video comms, zoom lenses, and map the environment as you go etc. etc.

  • Pasi Ripari

    torille

  • dk