Researchers Exploit Natural Quirk of Human Vision for Hidden Redirected Walking in VR

36

Researches from Stony Brook University, NVIDIA, and Adobe have devised a system which hides so-called ‘redirected walking’ techniques using saccades, natural eye movements which act like a momentary blindspot. Redirected walking changes the direction that a user is walking to create the illusion of moving through a larger virtual space than the physical space would allow.

Update (4/27/18): The researchers behind this work have reached out with the finished video presentation for the work, which has been included below.

Original Article (3/28/18): At NVIDIA’s GTC 2018 conference this week, researchers Anjul Patney and Qi Sun presented their saccade-driven redirected walking system for dynamic room-scale VR. Redirected walking uses novel techniques to steer users in VR away from real-world obstacles like walls, with the goal of creating the illusion of traversing a larger space than is actually available to the user.

There’s a number of ways to implement redirected walking, but the strengths of this saccade-driven method is that it’s hidden from the user, widely applicable to VR content, and dynamic, allowing the system to direct users away from objects newly introduced into the environment, and even moving objects, the researchers say.

The basic principle behind their work is an exploitation of a natural quirk of human vision—saccadic suppression—to hide small rotations to the virtual scene. Saccades are quick eye movements which happen when we move our gaze from one part of a scene to another. Instead of moving in a slow continuous motion from one gaze point to the next, our eyes quickly dart about, when not tracking a moving object or focused on a singular point, a process which takes tens of milliseconds.

SEE ALSO
These Third-party Vision Pro Controllers Just Hit Kickstarter, Promising to Unlock VR Gaming

An eye undertaking regular saccades

Saccadic suppression occurs during these movements, essentially rendering us blind for a brief moment until the eye reaches its new point of fixation. With precise eye-tracking technology from SMI and an HTC Vive headset, the researchers are able to detect and exploit that temporary blindness to hide a slight rotation of the scene from the user. As the user walks forward and looks around the scene, it is slowly rotated, just a few degrees per saccade, such that the user reflexively alters their walking direction in response to the new visual cues.

This method allows the system to steer users away from real-world walls, even when it seems like they’re walking in a straight line in the virtual world, creating the illusion that the the virtual space is significantly larger than the corresponding virtual space.

A VR backpack allows a user at GTC 2018 to move through the saccadic redirected walking demo without a tether. | Photo by Road to VR

The researchers have devised a GPU accelerated real-time path planning system, which dynamically adjusts the hidden scene rotation to redirect the user’s walking. Because the path planning routine operates in real-time, Patney and Sun say that it can account for objects newly introduced into the real world environment (like a chair), and can even be used to steer users clear of moving obstacles, like pets or potentially even other VR users inhabiting the same space.

The research is being shown off in a working demo this week at GTC 2018. An academic paper based on the work is expect to be published later this year.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • brubble

    Not necessarily new but very interesting to read a more technical study.

    • Ca1ibos

      Re-directed walking isn’t new but redirected walking hidden with eye saccades is. By the sounds of it, it could mean that a more severe rotation of the world can be hidden from the user than via older re-directed walking methods, thus allowing viable re-directed walking in much smaller Playspaces. However its still somewhat moot outside of VR arcades if the smallest playspace this technique allows is still larger than what the average VR gamer has available at home.

      • Tomas Sandven

        I can imagine that when VR gets more and more mainstream, many houses will be built around the possibility of a large VR play area. I’m going to build a house in a few years, and I’m definitely building a huge central room to serve as a living room / open kitchen, doubling as a large VR play area. I’m thinking something like 15 x 10 meters of unobstructed space, once I move the couch.

        Man I can’t wait :D

        • johngrimoldy

          Sounds awesome! I could see getting really excited about what you have planned. Keep in mind, that kind of construction will only be relevant for a short time; until the next large evolution of the technology. Eventually, having an oversized VR-dedicated playspace will seem quaint, like 12-foot satellite dishes.
          Still though, you can ALWAYS reuse the space when that eventuality happens.

          • Tomas Sandven

            Yeah I don’t really picture a large, open house falling out of style any time soon :P

  • Impressive. It is like hacking our brain. I wonder if there are any short or long term side effects outside of VR after using this process for long periods of time, also for younger people who are still developing their visual pathways.

  • Jean-Sebastien Perron

    Wow! This is brilliant.

    • Robert Morris

      Google pays all people $99/hr, also get week by week salary … any individual can also avail this!!on Friday I bought a gorgeous Lotus Elan after I been earning $6393 this four weeks .it is the extraordinary approach however you wont forgive yourself if you do not check it.!sg72p:=>=>=> http://GoogleLinksBestFreelanceInternetJobs/make/$99/everyhour ♥♥♥k♥♥e♥♥♥x♥♥♥v♥i♥♥♥h♥y♥p♥f♥♥m♥♥♥z♥♥♥x♥p♥♥q♥l♥l♥m♥m♥♥♥a♥♥♥t♥♥♥e♥♥♥r♥p♥♥t♥l:::::!eg123i:imkejch

  • Marcus Childs

    I have vision issues and wonder how those of us with messed up eye sight will work with such a system. Will it be helpful or harmful?

    • NickCoombe

      I imagine it would still work with you since the saccadic effect happens in the brain, not the eye, afaik. Unless your condition directly affects saccadic suppression, then it should work.

    • WyrdestGeek

      It seems like a fair question to me.

      You might be fine. You might not.

      A vision expert (e.g. an opthamologist or some such) could make a better guess.

  • Duane Aakre

    Wow. So if you take this and combine it with the light-field cameras from Google’s recent demo, I can envision a walking tour of places like Rome and Paris. Very exciting.

    • While you’re at it, throw their new 18 megapixel OLED and some foveated rendering to the headset and you’ve got a pretty nice VR system.

      • Jerald Doerr

        So true!! Took me a second to understand all of what’s going on but that’s just awesome!!!

    • mirak

      Yeah and you will be able to avoid bad virtual drivers you usually find in Paris.

  • Genius idea

  • Adderstone VR

    Brilliant idea. Won’t this cause your eye to ever so slightly miss the intended gaze target on every single movement though – I’d like to know if there are any side effect to having your eyes constantly miss their target and have to readjust?

    • Tomas Sandven

      Your eyes already slightly miss and re-adjust constantly anyway. You don’t move your eye 30 degrees and land right on your target. I can only speak for myself, but my eye jumps most of the way there in one leap, then do several small, quick hops to hone in on the target.

      My bet is they can safely move the world by a degree or two when your eye moves a significant distance, as long as they don’t rotate by more than x degrees every few seconds. The “intensity” of this system should probably be configurable though, since everybody is different.

      • Jerald Doerr

        Well from what I gather they can move the whole world on say X and Y buy like a foot and also rotate the camera a certain amount of degrees so there’s no warping of the image.

    • Gus Bisbal

      Your not actually looking at things continuously at any point in time.. You are mentally building a map of the world and your eyes are updating the map. Its is not a visual stream from your eyes. This is why peripheral vision is so important. When your eye moves, it literally stops working. No data is sent to the brain. It then re-sends it with the features in the scene as the reference points taken from your mental map of your environment. This technology uses that moment of temporary blindness to move the map. The re-targeting happens every single time you move your eye. Its just that in normal life the scene doesn’t move. Actually even that is wrong, when you run the whole world is moving a hell of a lot more than is being done in this tech. This is actually genius based on hard science and it is how its all going to work in the next generation of software.

  • Tomas Sandven

    This is so damn cool. I can imagine this combined with super large play areas (like a warehouse) to make practically infinite play area.

  • WyrdestGeek

    This is cool.

  • pogo

    FYI: This idea ist not new. It was already published in a research paper in 2015 by scientists from University of Münster, Germany: https://www.ncbi.nlm.nih.gov/pubmed/26357105

    • Peter Hansen

      So the difference seems to be the eye tracking method (camera-based vs. electro-physiology).

  • Eelke Folmer

    Any type of redirection requires knowing the exact path the user will take in advance which is impossible to predict for open environments.

    • Peter Hansen

      You clearly see in the data shown that this method is not perfect. Sometimes the intended physical boundaries are crossed. Nevertheless, this is the first implementation of redirected walking I would deem promising for smaller setups.

  • Peter Hansen

    So this is basically a form of Inattentional Blindness. Very nice! I wonder whether blinks would also work.

  • ziplock9000

    Very clever but I wonder if the user will feel weird as if something just isn’t right.

    • mirak

      Obviously if you are aiming a gun at something while walking, it will clearly move away from the sight.

  • VASQUAAL

    So, this is basically Star Trek’s Holodeck ?

  • Miqa

    While this is very neat, even in their large space and moving slowly, users seemed to reach the edge of the area. You would need very specific experiences and games for this to have any implications for consumers I think.

  • Nathanael Mooth

    Wow. This could really reduce the need for omni-directional treadmills for home use.

  • The demo should be released along w/ this paper. Why must they make it only at a venue? What they afraid of?

    Lazy…

    • Konchu

      Probably hardware since it sounds like it needs eye tracking. Even if people have it may not be easily configurable yet at least not at a shareable level.

  • Jerald Doerr

    Hopefully, after using this for an hour or two all us VR users won’t be stuck walking in circles in our room scale VR environment trying to get out! Ahhhh the doors over there!!!

  • mirak

    They could put SMS on a virtual smartphone that you read when you walk.
    This way you won’t even bother look the world when you walk.
    They can steer it 90°, you won’t even know it xD