Researchers Electrically Stimulate Muscles in Haptic Designed for Hands-free AR Input


Researchers at The Human Computer Interaction Lab at Hasso-Plattner-Institut in Potsdam, Germany, published a video recently showing a novel solution to the problem of wearable haptics for augmented reality. Using a lightweight, mobile electrical muscle stimulation (EMS) device that provides low-voltage to arm muscles, the idea is to let AR headset-users stay hands-free, but also be able to experience force-feedback when interacting with virtual objects, and feel extra forces when touching physical objects in their environment too.

Using a HoloLens headset, researchers show their proposed solution in action, which is made up of a backpack, a laptop computer running Unity, a battery-powered EMS machine, electrode pads, and visual markers to better track hand gestures. The researchers say their system “adds physical forces while keeping the users’ hands free to interact unencumbered.”

image courtesy Hasso-Plattner-Institut

Both HoloLens and the upcoming Magic Leap One include a physical controller; HoloLens has a simple ‘clicker’ and ML One has a 6DoF controller. While both systems admittedly incorporate gestural recognition, there’s still no established way for AR headset users to ‘feel’ the world around them.

According to the paper, which is being presented at this year’s ACM CHI Conference in Montréal, the EMS-based system actuates the user’s wrists, biceps, triceps and shoulder muscles with a low-voltage to simulate a sort of ‘virtual pressure’. This perceived pressure can be activated when you interact with virtual objects such as buttons, and even physical objects like real-world dials and levels to create an extra sense of force on the user’s arms.

There are some trade-offs when using this sort of system though, making it somewhat less practical for long-term use as it’s configured now. Two of the biggest drawbacks: it requires precise electrode placement and per-user calibration before each use, and it can also cause muscle fatigue, which would render it less useful and probably less comfortable.

Meta's New Demo Shows Players & Devs What Compelling MR Can Look Like

But maybe a little muscle stimulation can go a long way. The paper discusses using EMS sparingly, playing on the user’s keen sense for plausibility while in a physical (and not virtual) environment.

“In the case of [augmented reality], we observed users remarking how they enjoyed nuanced aspects of the EMS-enabled physics, for instance: “I can feel the couch is harder to move when it is stopped [due to our EMS-based static friction]”. As a recommendation for UX designers working in MR, we suggest aligning the “haptic-physics” with the expected physics as much as possible rather than resorting to exaggerations.

It’s an interesting step that could prove effective in a multi-pronged approach to adding haptics to AR wearables, the users of which would want to stay hands-free when going about their daily lives. Actuator-based gloves and vests have been a low-hanging fruit so far, and are quickly becoming a standard go-to for VR haptics, but still seem too much of a stretch for daily AR use. Force-feedback exoskeletons, which stop physical movements, are much bulkier and are even more of a stretch currently.

There’s no telling what the prevailing AR wearable will be in the future, but whatever it is, it’s going to have to be both light and useful—two aspects EMS seems to nail fairly well out of the gate.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.

Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 3,500 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • b8e5n

    This cannot replace physical presure of sking and real muscle force. Electrical stimulation is very unatural and thinking it could be natural is a joke… Not even mentionning the 2fps, the hand not covering the virtual objects. And what is the point in puting a filter on your frames? It’s nothing like it was highlighing it…
    Honestly, i think hololens is overrated and is help by very good marketing!

    • It isn’t about feedback that feels real, it’s about feedback that exist at all. This is better than nothing, and I think our brain can get used to good engieenierd system based on electrical stimulatiom. The frame is just an example, it could be anything, it’s just fot the demo.

  • JJ

    yesh they really need to work with Leap motion to get good hand tracking on the hololens. Right now it roughly picks up gesters like pinching and pulling but clearly is not tracking your hand or articulatin gthe fingers aside from their basic pinch gesture. If they had leap motions hand tracking software then things would be a lot cooler.

  • Lucidfeuer

    Interesting step toward body haptic interactions.

    • Firestorm185

      Interesting indeed. I’m more hopeful of exoskeleton-type solutions to these problems, but it’s interesting to see an electrical stimulation take on the problem.

      • Lucidfeuer

        Well I think this solution is actually more in-tune with an exoskeleton, than say full body suit which is never going to be adopted as the general solution for consumers. I like the idea that for mocap, body tracking and “haptics” feedback you just have a sort of harness that only cover the external points it needs and is really easy to put or remove.

  • Andrew Jakobs

    Unnatural stimulation of the muscles is something you really don’t want. It really is unhealthy to let ‘dummies’ use the electroshock therapy devices, cause that’s what they actually are. Leave those to the experts on handling them. Even the tellsell tummymusclebelts are unhealthy in the long run.

  • RJH

    The things that are possible now are amazing. Just think what the future holds for VR/AR. Wonderful.

  • I imagine the bugs while developing with the EMS… :P