Varjo, makers of high-end enterprise focused VR headsets, is adding real-time green-screen (aka chroma key) and marker tracking features to its XR-1 headset to make mixed reality applications more seamless.

Varjo’s XR-1 headset is a high-end enterprise development kit headset which combines the company’s retina-resolution ‘bionic display’ with a high quality, wide field of view, stereo camera for passthrough mixed reality. Today the company is rolling out new features in beta to make the headset’s mixed reality capability better and more flexible.

Just like the ‘green-screen’ filming technique, the XR-1 now supports real-time chroma keying, which allows a solid color in the real world to be seamlessly replaced with the virtual world.

As shown in the example below, this makes it easy (if you don’t mind setting up a green-screen) to show both the user’s body, a real flight stick, and even a physical pre-flight checklist, all surrounded by a virtual cockpit. Without the segmentation capability of the green-screen feature, the user’s body would be completely covered by the virtual world, and the user would need to operate the flight stick by feel and memory alone (far from ideal for training new pilots!).

This brute-force color replacement approach makes it easier, faster, and more accurate to segment the real world (especially the user themselves) from the virtual world compared to computer-vision approaches which require more processing power and time. The method also achieves high quality segmentation for distant objects, something which computer-vision approaches often struggle with.

On top of the chroma key capability, Varjo is also adding a simple marker tracking feature to the XR-1. Printed fiducial markers can be stuck to real-world objects to make them easy to track and replace with virtual objects.

Combined, the new features expand the XR-1’s capabilities for enterprises who see value in mixed reality functionality for things like training, design, research, and more. The features also dovetail nicely with Varjo’s impressive ‘Workspace’ concept which envisions an office environment which is part real and part virtual.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben Lang
Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Xron

    Ehh they want to show us how clear their lenses are and they are showing it in sd and hd quality, seems a bit “weird”…

    • Raphael

      The point isn’t how clear the lenses are. The point is to demonstrate live with the virtual world. We already know Varjo VR is super-high res and it’s not “how clear the lenses are”. It’s the overall spec rather than just clear lenses.

    • Adderstone VR

      It’s…it’s almost as if you READ the article

  • Raphael

    This is what I’d like to do with DCS world. Looks promising although the DCS vid didn’t match the virtual throttle position and he should have dressed for the part.

    For the DCS blending you’d also want an overhead light that mirrors the on-screen colour/brightness.

  • Rudl Za Vedno

    At least virtual future looks bright, when real world goes down the drain. Next stage is sitting at the VR table eating delicious lobster covering real world cockroach dinner on your plate.

  • mellott124

    I tried this at I/ITSEC 2019. Some of the best overlay AR/VR I’ve seen. Hand tracking was extremely low latency. Impressive stuff.

  • That’s overly cool. And I confirm that without an uniform background, the resulting segmentation is most of the times mediocre… for instance I’ve tried MixCast and the results without a green screen are… Mixed :D

  • Rupert Jung

    Two questions: Are positions tracked so your body can throw a virtual shadow? And could you add a lightstage to adapt the lighting of the body to the virtual environment?