Andrew Stern doesn’t enjoy most AAA video games because he wants to be able to say anything at any moment within a social simulation and be a participant in an interesting story. About once week, he’d like to engage within a high agency, interactive drama with artificially intelligent NPCs. Rather than long and extended play sessions, he’d prefer a short 20-30 minute experience that he play over and over again trying different strategies with characters who feel real and plausible.
This isn’t just a pipe dream because in 2005 Andrew was a co-creator of Façade, which is one of the only interactive drama games that has natural language input and offers both local and global agency to the player. For the past couple of years, Andrew has been working with his Façade collaborator Michael Mateas as well as with Larry LeBron on a DARPA-funded AI program. IMMERSE is a gestural-based, virtual training simulation for soldiers to learn deescalation social skills in non-English speaking environments. I had a chance to catch up with Andrew and Larry to learn more about using AI to create plausible characters, IMMERSE, as well as their new company called Playabl where they’re continuing to develop their Unity AI toolkit for creating fully interactive dramas.
LISTEN TO THE VOICES OF VR PODCAST
When I saw a demo of IMMERSE at the Portland Virtual Reality Meetup, I feel like I was seeing the future of what’s possible in creating plausible NPC characters. Andrew, Larry, and Michael created a number of different layers of social behavior modules that could be turned on and off. The combination of these modules started to yield emergent behaviors that transcend what would be possible in trying to hard code in all of the hundreds of potential branches.
In thinking about Presence in VR, according to Mel Slater’s theory, the two major ingredients of The Place Illusion and the Plausibility Illusion. Creating plausibility within a VR experience means that you have to create an environment that feels coherent, and believable. Rob Morgan says that this means that NPCs in VR need to be responsive to your actions even more so than in a 2D game. Ross Mead talks about all of the different body language and social behaviors that are important to this. And in terms of natural language input, Façade was a pioneer of allowing you to say anything at anytime and have the story adapt and respond to your character.
I think that this type of AI work and characters are going to be a huge part of creating experiences where you’re a “Character with Impact.” For anyone interested in learning more about how to architect and create a interactive drama with both local and global agency, then I’d highly suggest spending $5 to buy the “Behind the Façade” Guide, which is an amazing document that talks about the architecture of a highly dynamic story that makes the user feel like they’re an active participant in creating it.
Andrew, Larry, and Michael are still in the early phases of Playabl, but I look forward to what types of tools and stories that they end up creating. For people interested in moving beyond passive experiences, then their AI technology and framework is one of the most compelling approaches to high agency, interactive drama that I’ve seen so far.
Become a Patron! Support The Voices of VR Podcast Patreon
Theme music: “Fatality” by Tigoolio
