IBM Watson, the artificial intelligence platform designed to understand natural language, today launched support for Star Trek: Bridge Crew (2017) across PSVR, Oculus Rift and HTC Vive.

Before the service launched today, lone players could control the ship’s other posts—Engineering, Tactical, Helm—by clicking a few boxes to issue orders. Now a sole captain (also with a mixed crew of humans and non-humans) can complete whole missions by issuing commands directly to the non-human-controlled characters using natural language.

image courtesy IBM

Voice commands are enabled by IBM’s VR Speech Sandbox program, which is available on GitHub for developers to integrate speech controls into their own VR applications. The Sandbox, released in May, combines IBM’s Watson Unity SDK with two services, Watson Speech to Text and Watson Conversation.

at the Captain’s chair, image captured by Road to VR

We had a chance to go hands-on at E3 2017 with Star Trek: Bridge Crew embedded with the Watson-powered voice recognition, a feature that’s initiated during gameplay with a single button press. While talking directly to your digital crew does provide some of those iconic moments (“Engage!” and “Fire phasers!”), and most orders went through without a hitch, Watson still has trouble parsing some pretty basic things. For example, Watson doesn’t understand when you use the name of ships, so “scan the Polaris” just doesn’t register. Watson also didn’t pick up on a few things that would seem pretty easy at face value. Commands like “fire on the target”, “fire on the enemy,” and “come on, let’s warp already!” fell on deaf digital ears.

SEE ALSO
PlayStation E3 Keynote Puts Blockbusters Center Stage, Relegating VR to the Sidelines

IBM says their VR speech controls aren’t “keyword driven exchanges,” but are built around recognition of natural language and the intent behind what’s being said. Watson also has the capacity to improve its understanding over time, so those “Lets get the hell out of here, you stupid robots!” may actually register one day.

This however doesn’t stop a pretty weird logical disconnect that occurs when talking to a bot-controlled NPC, and it stems from the fact that I was at first imbuing the NPCs with actual intelligence. When talking directly to them, I was instinctively relying on them naturally to help me do my job, to have eyes and ears and not only understand the intent of my speech, but also the intent of the mission. A human tactical officer would have seen that we were getting fired on, and I wouldn’t have had to issue the order to keep the Bird of Prey within phaser range. I wouldn’t have to even select the target because the Helmsman would have done it for me. IBM isn’t claiming to be able to do any of that with its cognitive computing platform, but the frustration of figuring out what Watson can and can’t do is a stark reality, especially when getting your tail-end blasted out of the final frontier.

SEE ALSO
IMAX And Ubisoft Launch New Mission for 'Star Trek: Bridge Crew' at IMAX VR Centers

In the end, Watson-supported voice commands may not be perfect—because when the Red Shirts are dropping like flies and consoles are exploding all over the place, the last thing you want to do is take the time to repeat an important order—but the fact that you can talk to an NPC in VR and get a pretty reliable response is amazing to say the least.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • SomeGuyorAnother

    Voice commands is the next and easiest step towards immersion in VR. Reading aloud dialogue options to select them, issueing commands such as this, or literally shouting your dragon shouts in Skyrim would add much to how games are played.

    • Sam Illingworth

      Shouting the shouts – OMG YES!!

      They could totally do that too, that’s surely well within current technological limits! Your idea for dialogue would be ace too.

      • SomeGuyorAnother

        There was actually an early Skyrim mod when they introduced Steam Workshop that did this. Clunky at times, but still conceptually brilliant.

        • kEwLiO

          The Xbox version had Kinect voice support that allowed you to do this… o-o

  • John Horn

    IBM Watson is probably the only feasible way of doing it properly, thanks to its deep self-learning algorithms. But it will still not come close to a real human’s ability to understand situational context etc.

  • Get Schwifty!

    While I have faith ultimately in AI to bring a trans-formative change about, I still think people have overly rosy ideas about its real application. A day doesn’t go by on CNBC for instance where the topics of robotics and AI’s disruptive influence is bearing down on us but I think they are to a large extent reiterating the marketing they are getting from IBM, Intel, etc. All the while, simple concepts in voice like “Scan the Polaris” aren’t picked up by one of the leading AI systems out there. Are you sure you are really ready to trust AI-driven cars? I know I don’t, I don’t care how deep self-learning they claim to be.

    • >>Are you sure you are really ready to trust AI-driven cars?

      Only when ALL cars are AI would I trust it. There is no AI that can predict the randomness and stupidity of human interaction, we always find a way to break something. Saying that we have unknowingly trusted auto-pilot in airplanes for years.

      • Get Schwifty!

        Agreed – though planes are far more trivial than cars as a problem for controls because the conditions are magnitudes simpler to deal with.

  • Ben Blood

    The voice commands work pretty well. Not as well as I had hoped with the Watson branding… But still fun for solo play, and sometimes helpful in multiplayer. Playing all 5 episodes now so see if I can do them all straight voice commands and not switching positions…

    https://www.youtube.com/watch?v=nd4BCp0-dRI

  • Kev

    Would be epic to add full voice interaction to Skyrim or fallout NPC’s. A person below is mentioning the Skyrim shouts and that would be great as well. But full AI interaction for in game NPC’s would truly change RPG’s forever.

  • ummm…

    i ahve the game but haven’t played in a few weeks. how is the support?