Valve’s resident experimental psychologist Dr. Mike Ambinder took the stage at GDC 2019 today for his talk on the state of brain-computer interfaces (BCIs) and how they’ll inform the future of game design. In his talk, Ambinder put forth a near future where VR/AR headsets are kitted with non-invasive electroencephalogram (EEG) devices that could one day provide data to game designers so they can create a new generation of smarter, more reactive games. The long-term view is definitely trending towards neuronal implants, Ambinder says, but we’re not there yet.

Ambinder explained that many of the traditionally measurable ‘bio-feedback’ such as heart-rate, galvanic skin response, eye movement, facial expression, muscle tension, and physical posture are all important to understanding what’s happening to the player at any given moment, although he argues that reaching into the core of the matter, the brain itself, is the frontier that holds the most promise for game developers.

Brain-computer interfaces are basically a communication method that translates neuronal signals into actionable input for computers. In the context of gaming, the Ambinder’s envisioned goal is to acquire physiological data from a player and use that to inform the game whether a player is happy, sad, frustrated, bored, focused, distracted, etc; the idea is to figure out why that happens in each circumstance—be it anger from experiencing a game-breaking bug, or happiness when a specific goal is achieved—and build systems to leverage this data.

And it’s not as far-fetched as you may think, Ambinder maintains:

“We can measure responses to in-game stimuli. And we’re not always getting [data] reliably, but we’re starting to figure out how. Think about what you’d want to know about your players. There’s a long list of things we can get right now with current technology, current generation analysis, and current generation experimentation,” he said.

Valve’s Mike Ambinder donning an OpenBCI EEG helmet, Image captured by Road to VR

Moment-to-moment insights into the user’s cognitive state could provide a sort of adaptive, immersive gameplay which could include variable game difficulty, on-the-fly AI responses tailored to the player’s mindset, and even replacing traditional input systems all together—all within the purview of BCI’s gaming future, Ambinder said.

In the near-term, Ambinder sees EEG as the easiest way to start collecting that sort of psychological data and organizing it into something actionable for developers to create more immersive games. In the case of EEG, electrodes are non-invasively placed on various points of the scalp, and used to measure voltage fluctuations of the brain’s neurons. Things like attention, learning, memory, and intention have been measured in some respects using EEG, and Ambinder hopes these things, which have previously been the subject of scientific research, will make their way into mainstream game design at some point. But people aren’t just going to wear a full 35-electrode location EEG helmet like the one from the OpenBCI initiative on its lonesome though; that’s where AR/VR comes in.

Rift S Isn't the Headset Fans Want, But Facebook Wagers It's What Their Ecosystem Needs

“If you’re going to measure brain signals, you need a way to get people to wear a helmet. If only us as game designers had a way of doing that,” Ambinder said jokingly, showing a slide of an HTC Vive headset. “One advantage of AR and VR as well is you’re getting consistent contact with the source of brain activity. So you might be able to do interesting things if you could convince them to wear a helmet with EEG sensors.”

Companies like Neurable have already begun productizing EEG devices especially built to work with VR devices, letting people “control software and devices using only their brain activity,” the company claims.

That said, EEG data isn’t a perfect solution. Ambinder compared it to sitting outside of a football stadium and trying to figure out what’s happening on the field just by listening to the intensity of the crowd’s reaction. The current generation of BCI devices are noisy, and EEG is one of the noisiest due to its job of picking up neuronal signals through the skull, scalp, and hair.

Image captured by Road to VR

There’s still plenty to learn from EEG, Ambinder says, although deeper knowledge and a push towards more immersive, adaptive games would likely necessitate invasive brain implants. Before that happens, they’d need to pass what he calls the ‘Lasik threshold’ though—a procedure that is technically invasive but confers enough potential benefits to the user while minimizing the procedure’s overall risk.

Ambinder had nothing to announce regarding the company’s plans for potentially kickstarting BCI with something as monumental as a consumerized virtual reality-EEG headset (or brain implants for that matter), although he sees EEG as a definite early avenue for capturing the sort of large data sets needed to start building out interaction systems that could span everything from creating better tutorials by learning about the user’s individual ability to learn, to making games capable of learning how players want to play as opposed to how they should play.

Making this knowledge more granular (and accurate) is still in its early phases, something that needs both sufficient time and a large enough userbase to gather data across a wide set of people. There’s a long list of powerful uses and equally powerful pitfalls yet to come.

If you want to get into the nitty gritty, Ambinder’s full talk will be up on the GDC’s YouTube page in the coming days. We’ll toss the video down here when it comes, so check back soon.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • DaKangaroo

    Thanks Valve. =)

  • Muzufuzo

    Neural interfaces or BCIs are definitely the way to go. I don’t see any great future for VR if these ideas won’t be realized. There is only so much that can be done with motion sensors.

    • q23main

      Yes, but how will I get fit if I’m not saberin’ those beats with my bare hands?

  • Bob

    But can the brain run Crysis? :)

    • TJ Studio

      Why do you even want the thing to run Crysis? It won’t happen!

      • ines

        The latest analysis shows that approximately 75% people are involved into internet based tasks. On line arena is becoming bigger and more beneficial and delivering plenty of work at home opportunities. Home based on-line tasks are trending and changing individual’s everyday lives. The key reason why it is really widely used? Mainly because it lets you do the job from anywhere and any time. You get more time to allocate with all your family and can plan out tours for getaways. Persons are generating great earnings of $32000 every single week by utilizing the effective and intelligent techniques. Carrying out right work in a right path will always lead us towards success. You will start to get paid from the 1st day when you browse through our web site. >>>>> REGISTER

      • Rail

        A NVlink of 2 2080ti can run Crysis(last One) at solid 60fps…lol

    • dk

      have u ever had a dream :P

      • toi

        Recent analysis demonstrates that more than 75% people are engaged into web-based jobs. On line world-wide is simply becoming bigger and better and giving plenty of make money online opportunities. Work from home internet jobs are trending and changing individual’s everyday lives. The reason why it is popular? Simply because it lets you do the job from anywhere and anytime. You will enjoy more time to allocate with all your family members and can plan out tours for getaways. Individuals are making wonderful revenue of $26000 each and every week by utilizing the efficient and smart methods. Doing right work in a right direction will definitely lead us in the direction of success. You will start to get paid from the 1st day when you look at our web-site. >>>>> HURRY

        • dk

          cool a bot

  • Vegeta785

    Pretty cool if I do say so myself.

  • Get Schwifty!

    This is your brain; this is your brain on Steam….. talking BCIs and yet we still have wands with Vive…. LOL…

    • doug

      Yes, vive’s wands are immersive only for gun grips, sword hilts, axe handles, bow grips, etc. How is a player supposed to flash a rad gang sign in multiplayer with a vive wand… HTC should have thought of that. ;-)

  • Michael Slesinski

    we already have bci’s theyre called hands.
    more seriously though until theres a release date they should probably just shut the fuck up about it. someday “soon” we will have flying cars, men walking on mars, the sun wiping out all life on earth.. and ofcourse HALF LIFE FUCKING 3!

    • Vegeta785

      You’re right.

  • Pretty interesting! Thank you for sharing this.
    Virtual Reality Social

  • Michael Jacobs

    Nope. This is where I draw the line. For no reason am I letting any company the ability to read any of my brain brain activity, in any form. I don’t care how amazing it’ll make games. I really don’t care about the advantages if I have to let something see inside my head. I bet Facebook is jacking off to this idea.

    • asdf

      then u are stupid.