Disney Research Shows How VR Can Be Used to Study Human Perception


Researchers from Disney use a virtual reality headset and a real ball tracked in VR to understand the perceptual factors behind the seemingly simple act of catching a thrown object. The researchers say the work sets a foundation for more complex and meaningful dynamic interaction between users in VR and real-world objects.

Disney Research, an arm of The Walt Disney Company, does broad research that applies across the company’s media and entertainment efforts. With Disney known as being one of the pioneering companies to employ VR in the out-of-home space prior to the contemporary era of VR, it should come as no surprise that the company continues to explore this space.

vr-ball-catch-disney-researchIn a new paper from Disney Research a VR headset is used in conjunction with a high-end tracking system to recreate the simple experience of catching a real thrown ball in VR. But why? With the experience being simulated in virtual reality, the researchers are free to easily modify it—in ways largely impossible in real life—as they see fit to study different aspects of the perception and action of catching a thrown object.

In this case, researchers Matthew K.X.J. Pan and Günter Niemeyer started with a visualization that showed just a ball flying through the air as you’d see in real life. Using VR they were able to add a virtual trajectory line to the ball’s path, and even remove the ball completely, to see what happened when the user had only the trajectory line to rely on. The video heading this article summarizes the tests they performed.

OptiTrack's Precise 'Void' Style Tracking Lets You Play Real Basketball in VR

Perhaps most interesting is when they removed the ball and the trajectory line and only showed the user a target of where the ball would land. Doing this lead to a distinct difference in the user’s method of catching the ball.

Whereas the ball and trajectory visualization resulted in a natural catching motion, with only the target to rely on, the user switched to what the authors described as a more “robotic” motion to align their hand to where the ball would land, and then wait for it to arrive.

This suggests that our brains don’t simply calculate the endpoint of the trajectory and then move our hand to the point to catch it, like a computer might; instead it seems as if we continuously perceive the motion of the ball and synchronize our movements with it in some way. The authors elaborate:

20 [of 132] of these tosses were made with only the [visualization of the] virtual ball which most closely matches how balls are caught in the physical world. In this condition, 95% of balls were caught, indicating that our system allows users to catch reliably. Video and screen capture footage indicate that during the catch, the user visually focuses on the trajectory of the ball and does not keep their hands within viewing range until just before the catch. From this evidence, it can be inferred that prioprioception is used to position the hands using visual and depth cues of the ball.

Catching with the other visualizations did not seem to affect catching behaviors, except in the cases where the virtual ball was not rendered: the removal of the virtual ball from the VR scene seems to allow the catcher’s hands to reach the catch lo­cation much earlier prior to catching. The most apparent expla­nation for this phenomenon lies with the observation that the user is forced to alter catching strategy: the catcher has to rely on the target point/trajectory and so the motor task has changed from a catching task, which had required higher brain functions to estimate the trajectory, to a simpler, visually ­guided pointing task requiring no estimation at all.

The researchers say that their work aims to study how people can interact dynamically with real objects in VR, with this first study of a simple task laying the groundwork for more complex interactions:

…combining virtual and physical dynamic interactions to enrich virtual reality experiences is feasible. […] We believe this work provides valuable insight which informs how interactions with dynamic objects can be achieved while users are immersed in VR. As a result of these preliminary findings, we have discovered many more avenues for future work in dynamic object interactions in VR.

Which in research talk means, ‘you better bet we’ll be studying this further!’

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.

Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • OgreTactics

    Wow, this is incredibly precise. We too often underestimate the brain horse power behind Disney not just in tech, but also in terms of policies and long-term positioning: never have I ever seen, at least in the last decade, any subtle racist, sexist, homophobic, war-mongering, or ideological bias of any sort hidden between the lines in their movies even though Hollywood movies are riddled with them.

    • Andrew Jakobs

      than you haven’t seen many disney movies.. those movies are just as riddled with it as from other studio’s..

      • OgreTactics

        No they’re not. Well i’m mainly talking about Pixar, but incredibly rarely enough, they’re not “riddled” with it unlike most other Hollywood or TV production.

  • Very powerful experiment. I did not understand the point of the video until it reached the point of the “target only” visualization… and then I was really excited by it

  • Erik

    Not sure what the purpose of this experiment is. Of course you’re gonna react faster when you don’t know when the ball will land in your hand. When you see the ball you can estimate when it will land and therefore take more time to move your arm.