Regina Dugan, VP of Engineering at Facebook’s Building 8 skunkworks and former head of DARPA, took the stage today at F8, the company’s annual developer conference, to highlight some of the research into brain-computer interfaces going on at the world’s most far-reaching social network. While it’s still early days, Facebook wants to start solving some of the AR input problem today by using tech that will essentially read your mind.

6 months in the making, Facebook has assembled a team of more than 60 scientists, engineers and system integrators specialized in machine learning methods for decoding speech and language, in optical neuroimaging systems, and “the most advanced neural prosthesis in the world,” all in effort to crack the question: How do people interact with the greater digital world when you can’t speak and don’t have use of your hands?

Facebook’s Regina Dugan, image courtesy Facebook

At first blush, the question may seem like it’s geared entirely at people without the use of their limbs, like those with Locked-in Syndrome, a malady that causes full-body paralysis and inability to produce speech. But in the realm of consumer tech, making what Dugan calls even a simple “brain-mouse for AR” that lets you click a binary ‘yes’ or ‘no’ could have big implications to the field. The goal, she says, is direct brain-to-text typing and “it’s just the kind of fluid computer interface need for AR.”

While research regarding brain-computer interfaces has mainly been in service of these sorts of debilitating conditions, the overall goal of the project, Dugan says, is to create a brain-computer system capable of letting you type 100 words per minute—reportedly 5 times faster than you can type on a smartphone—with words taken straight from the speech center of your brain. And it’s not just for the disabled, but targeted at everyone.

SEE ALSO
This Open World VR Game is Still Ahead of Its Time – Inside XR Design

“We’re talking about decoding those words, the ones you’ve already decided to share by sending them to the speech center of your brain: a silent speech-interface with all the flexibility and speed of voice, but with the privacy of typed text,” Dugan says—something that would be invaluable to an always-on wearable like a light, glasses-like AR headset.

image courtesy Facebook

Because basic systems in use today now don’t operate in real-time and require surgery to implant electrodes—a giant barrier we’ve yet to surmount—Facebook’s new team is researching non-invasive sensors based on optical imaging that Dugan says would need to sample data at hundreds of times per second and precise to millimeters. A tall order, but technically feasible, she says.

This could be done by bombarding the brain with quasi-ballistic photons, light particles that Dugan says can give more accurate readings of the brain than contemporary methods. When designing a non-invasive optical imaging-based system, you need light to go through hair, skull, and all the wibbly bits in between and then read the brain for activity. Again, it’s early days, but Facebook has determined optical imaging as the best place to start.

The big picture, Dugan says, is about creating ways for people to even connect across language barriers by reading the semantic meanings of words behind human languages like Mandarin or Spanish.

Check out Facebook’s F8 day-2 keynote here. Regina Dugan’s talk starts at 1:18:00.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 3,500 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Get Schwifty!

    Okay, who wants to be the first to say “I hate Facebook, therefore I am going to have nothing to do with them or this effort which is clearly an attempt to steal your memories and thoughts”? Oh wait…

    • level-up

      i don’t like this
      I love VR now, I move naturally

    • Wednaud Ronelus

      You can’t stop the time my friend. We will cross the rubicon whether you like it or not.

    • Get Schwifty2

      “I love Facebook, therefore I am going to preemptively post a sarcastic message to deter any hate aiming at my beloved company”

  • TechniGuest

    Facebook is jumping on the AR bandwagon. I wonder if this has anything to do with their recent shifts in VR departments. Could be a good indicator that pulling resources from VR and into AR.

  • Zerofool

    Personally, I’m more interested in what Elon Musk’s latest project – Neuralink – will achieve in this regard. I eagerly await the blog post on WaitButWhy which will focus on that new company (and BCI as a whole).

  • Foreign Devil

    I’m satisfied with my current brain computer interface. . .namely my hand, eyes and voice.

  • Nigerian Wizard

    Ill be able to install Adblock in my brain in my lifetime. How exciting.

    “You will live to see man made horrors beyond your comprehension” -Nikola Tesla

  • DaKangaroo

    As a 3D artist, I can’t describe how much I’d love to have an interface with my computer that lets me create anything I can imagine in my head. I’d work easily 100x faster than I currently do, and create so many things, it’d be wonderful..

    But at the end of the day, what it comes down to, is asking me to put a device on top of my head that can literally read my thoughts. In order for me to feel comfortable wearing it, I’d have to feel confident that the device will not be hacked by hackers, can not in fact be hacked by hackers, that the device is secured and has foolproof safeguards, will inform me of everything that is being transmitted beyond the device, that the data sourced from the device won’t be sent to a corporation to be datamined for profit or to manipulate my buying habits, won’t be sent to spy agencies to monitor people’s thoughts..

    Do I trust an array of greedy corporations, top secret out of control government spy agencies, politicians, engineers and programmers to create a flawless system of total privacy that won’t be spied on, data mined or hacked?

    The answer is, sadly, no. I doubt I ever will.

  • Wow