In what seems like a total reversal of the usual paradigm, a VR researcher is showing the benefits of augmenting the virtual world with the real world.

In ‘traditional’ augmented reality, we take a scene from real life and overlay it with useful digital information that in some way enhances the experience.

Oliver Kreylos, a VR researcher from UC Davis, has turned the tables by bringing the real world into the virtual world for the same purpose.

Kreylos, who last told us about the important of proper stereoscopic 3D rendering, has adapted his custom VRUI Toolkit environment to work with the Oculus Rift and Razer Hydra. A recently released video shows the powerful data exploration that can be done in virtual reality:

While I find the ability to interact and analyze complex data to be very interesting, what really strikes me is what Kreylos does at  3:28 in the video. He augments the virtual world with the real world by bringing his desk into the VR environment using the Microsoft Kinect.

Due to the Kinect’s limitations, the results are a bit jittery, but the implications are clear: he can easily reach out and touch everything on the desk.  This is significant for Kreylos as he is used to using CAVE environments where you can see your hands.  With the ‘augmented virtual reality’ he doesn’t have to guess where his hands, the mouse, or the keyboard are because they now exist in the virtual world.

Kreylos has also used this technique to bring his entire body into a VR environment. Such capabilities is one of the many exciting possibilities that could be enabled with the inclusion of a depth-camera mounted to the Oculus Rift.

Alternative Text

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Finstern

    This is actually pretty awesome! I am also working on Kinect integration but more for positional coordinates for walking around inside a small shuttlecraft type ship!

    This really seems to be the first implementation of a Holodeck “Arch” in which the user can call up a more traditional control panel into the VR scene

  • Mageoftheyear

    Fracking earthquakes. Honestly, is there anything a Rifter won’t develop for?
    Very interesting demonstration.

  • Psuedonymous

    Can’t see the video due to work proxy, but this sounds like Augmented Virtuality: using physical ‘props’ embodies in a virtual environment.