Not Perfect, But Promising

Mask wasn’t perfect by any means. I saw the occasional misinterpretation of data which could cause my avatar’s face to do something that I wasn’t actually doing. When it came to normal blinking (not winking), the prototype didn’t seem to pick up my blinks very well. However, Tadi claims that the algorithm gets smarter and more robust over time individually for each user as they use it.

How much better it can get over time is the big question. It’s going to need to be pretty darn close to 100% because users will need to be able to trust the system to represent them accurately. Facial expressions are a hugely important part of unspoken communication; you wouldn’t want to be in an important virtual meeting with a potential new client and look as if you’re sneering at them (due to misinterpretation of your facial expressions) when you’re actually smiling. And even outside of important meeting scenarios, face-tracking in VR still needs to be highly robust so that it doesn’t lead to awkward facial movements that end up dropping your avatar into the uncanny valley.

The ability to improve Mask’s robustness from 90% to 99% will make or break the applicability of the tech.

An Unfair Advantage

While computer-vision based face-tracking has the potential to be much more precise—capturing subtle changes in the face rather than just being able to jump between a few poses—even if the headset didn’t pose a significant challenge, this electrode-based approach has an advantage that computer-vision based face-tracking physically cannot: prediction.

Tadi claims that the Mask algorithm can, via the electrical signals being read from your face, sense the expressions you’re going to make “20 to 30 milliseconds” before you make them. If true, that potentially gives an advantage to this method, opening the door to essentially instantaneous mirroring of your expression in the virtual world.

The MASK prototype currently requires electrodes attached to earlobes in addition to those that contact the face, though the creators say these would be built into on-headset headphones

The electrode/algorithm approach to face-tracking is also interesting because it doesn’t require much processing power, Tadi says, and the workload can easily be performance on a VR headset powered by a smartphone. That’s in contrast to the computer-vision approach which requires more heavy-duty processing to derive your facial expression from each frame.

– – — – –

So, Mask seems promising. What’s next? Tadi says that the underlying technology is compatible with any headset, and that the company is in talks with all the major players in the market. Ultimately the company seeks not to sell the hardware as an add-on, but offer a simple blueprint for direct integration and then license the algorithm. We could see a headset with this tech integrated as early as holiday 2017.

Eye-tracking Glasses Give Glimpse into Professional Pianist's Perception

While I think that VR is destined to one day incorporate face-tracking, it’s not yet clear to me exactly how precise and accurate it needs to be in order to be a must-have feature (and one that doesn’t risk misrepresenting the user). Given the current state of the industry, bringing headset costs down is a greater priority right now than adding more features (and cost), which could mean tough timing for Mask.

Alternative Text

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • psuedonymous

    I;m afraid this is indeed a case of ‘neuro-bamboozling’: what is going on here is absolutely NOT Electroencephalography, but ElectroMYOgraphy. They are measuring muscle contractions, not neuron activation. It;s double infuriating because their tech is functional and useful (and measuring muscle contractions is the correct way to do this), but they go and muddy the water by using Buzzword Bullshit to market it.

    • benz145

      They didn’t claim that this is “brain reading” technology, they were clear that the tech is measuring the electrical signals in your face and not reading your mind.

  • VRMaven

    Despite being a billion dollar startup, I wouldn’t hold your breath on them having anything on the market this year- especially with that dorky ear clip

  • wheeler

    Something like this mixed with eyetracking will take social VR experiences to the next level.

  • Strawb77

    “..look as if you’re sneering at them (due to misinterpretation of your facial expressions) when you’re actually smiling.”
    one application i can see with this would be with [ie:] autistic people, who have great difficulty reading [and presenting] facial expressions- if the mask could be made haptic, for instance, you could show what expressions `feel` like as you make them.
    possibly could help serious burns patients or others who have had skin grafts also.