Researchers from Disney use a virtual reality headset and a real ball tracked in VR to understand the perceptual factors behind the seemingly simple act of catching a thrown object. The researchers say the work sets a foundation for more complex and meaningful dynamic interaction between users in VR and real-world objects.
Disney Research, an arm of The Walt Disney Company, does broad research that applies across the company’s media and entertainment efforts. With Disney known as being one of the pioneering companies to employ VR in the out-of-home space prior to the contemporary era of VR, it should come as no surprise that the company continues to explore this space.
In this case, researchers Matthew K.X.J. Pan and Günter Niemeyer started with a visualization that showed just a ball flying through the air as you’d see in real life. Using VR they were able to add a virtual trajectory line to the ball’s path, and even remove the ball completely, to see what happened when the user had only the trajectory line to rely on. The video heading this article summarizes the tests they performed.
Perhaps most interesting is when they removed the ball and the trajectory line and only showed the user a target of where the ball would land. Doing this lead to a distinct difference in the user’s method of catching the ball.
Whereas the ball and trajectory visualization resulted in a natural catching motion, with only the target to rely on, the user switched to what the authors described as a more “robotic” motion to align their hand to where the ball would land, and then wait for it to arrive.
This suggests that our brains don’t simply calculate the endpoint of the trajectory and then move our hand to the point to catch it, like a computer might; instead it seems as if we continuously perceive the motion of the ball and synchronize our movements with it in some way. The authors elaborate:
20 [of 132] of these tosses were made with only the [visualization of the] virtual ball which most closely matches how balls are caught in the physical world. In this condition, 95% of balls were caught, indicating that our system allows users to catch reliably. Video and screen capture footage indicate that during the catch, the user visually focuses on the trajectory of the ball and does not keep their hands within viewing range until just before the catch. From this evidence, it can be inferred that prioprioception is used to position the hands using visual and depth cues of the ball.
Catching with the other visualizations did not seem to affect catching behaviors, except in the cases where the virtual ball was not rendered: the removal of the virtual ball from the VR scene seems to allow the catcher’s hands to reach the catch location much earlier prior to catching. The most apparent explanation for this phenomenon lies with the observation that the user is forced to alter catching strategy: the catcher has to rely on the target point/trajectory and so the motor task has changed from a catching task, which had required higher brain functions to estimate the trajectory, to a simpler, visually guided pointing task requiring no estimation at all.
The researchers say that their work aims to study how people can interact dynamically with real objects in VR, with this first study of a simple task laying the groundwork for more complex interactions:
…combining virtual and physical dynamic interactions to enrich virtual reality experiences is feasible. […] We believe this work provides valuable insight which informs how interactions with dynamic objects can be achieved while users are immersed in VR. As a result of these preliminary findings, we have discovered many more avenues for future work in dynamic object interactions in VR.
Which in research talk means, ‘you better bet we’ll be studying this further!’