Microsoft Research has devised two novel methods for more realistic haptic feedback on virtual reality controllers. They call it NormalTouch and TextureTouch.
Haptic feedback in general-purpose controllers has been limited to vibration feedback since the introduction of the Rumble Pak for the Nintendo 64 in 1997. Vibration motors come in all shapes and sizes, the most popular being the Eccentric Rotating Mass (ERM) motor, found most modern gamepads. Mobile phones often use very small ERM motors, or. more recently, linear actuators. Linear actuators tend to offer more haptic ‘detail’ and responsiveness, as can be found in Apple’s ‘Taptic Engine’, the HTC Vive controllers, an the Oculus Touch controllers. While vibrations as haptic feedback is the current state of the art in the consumer realm, limitations remain.
Tactile feedback has proven to be effective across a wide variety of applications, but if you’re looking for significant force or resistance in your haptics, you need kinesthetic feedback. This is commonly available through force-feedback controllers, which tend to be designed for a specific task, such as joysticks for flight/space simulators, and wheels for driving simulators. The wealth of powerful haptic hardware on the market is one of the main reasons why flight and driving simulations are already so effective in VR. The closest product to a general-purpose kinesthetic controller is probably still the Novint Falcon, first shown in 2006, but this is also fairly limited, as it needs to be attached to a desk.
Microsoft Research’s new experimental controllers bring kinesthetics into the VR space, offering two types of force-feedback applied to fully-tracked motion controllers. NormalTouch uses three servo motors to operate a small disc with tilt and extrusion movements, and TextureTouch uses a bank of 16 servos to operate a 4×4 pixel array of small blocks that move up and down to correspond to virtual shapes and structures. The result is a feeling of physical resistance as you drag your finger across a virtual shape, with enough fidelity to actually convey a sense of touch and an understanding of an object’s form and texture.
In both controllers, the feedback surface acts on a single finger or thumb, which may limit the practical use cases. But the key point is that this type of feedback is normally the domain of dedicated devices, elaborate gloves, or exoskeletons, whereas Microsoft Research’s designs are based on a normal handheld controller, which Michael Abrash, Oculus’ Chief Scientist, recently suggested could remain the standard input for VR for decades to come.
Texture is one thing, but offering real resistance (where the virtual world can push back on you) is still a pipe dream however, as there is nothing preventing the user from clipping through objects with today’s VR controllers. But with more realistic haptics, the desire to clip through something is reduced, in the same way that more realistic VR visuals often prevents people from trying to walk through virtual objects.
In their testing, the Microsoft Research team developed a ‘penetration compensation’ technique, that made it appear that the user’s hand was not clipping, by decoupling them from the real tracking location. The finger is the most sensitive part of the hand to kinesthetic feedback, so this is effective, although it remains to be seen how this haptic-visual mismatch could work in a less controlled environment.
The research group’s findings are promising even at this early stage. Three tests were run—targeting accuracy, tracing accuracy and fidelity assessment—and both controllers were used, comparing them to vibration-only feedback and visual-only feedback. Both new haptic feedback techniques demonstrated advantages over vibration and visual only tests, despite some of the limitations of the prototypes causing issues.
The designs can no doubt be improved dramatically in terms of ergonomics, range of movement, responsiveness and detail, but already testers reacted positively to the heightened sense of touch. It was noted that the fact it’s already effective demonstrates the overriding power of the visual system, and that perhaps fully detailed or accurate feedback isn’t too critical, as the visual system automatically makes the corrections.
Perhaps the toughest challenge of this project is in improving the physical design. Any device with a large number of mechanical parts always comes at a cost, usually in the form of weight and noise, and that’s certainly the case here. If this technology was utilized for a consumer product it would need to get smaller while staying quiet and reliable. It’s an area of research that is worth pursuing further, but it’s unclear at this stage how likely these prototype haptic technologies are to find their way into a real product.