The Marvel What If…? experience is set to launch tomorrow exclusively on Vision Pro. We got an early look at the experience and found a polished presentation that foreshadows the possibilities and pitfalls of immersive VR games on the headset.

Marvel’s What If…? on Vision Pro is being described as an “interactive Disney+ Original story.” And while it isn’t meant to be a full-blown VR game, it’s the headset’s biggest fully immersive production to date—most other native ‘games’ on the headset feel a lot like board games that float in front of you.

In What If…? players experience a narrative blended with casual gameplay driven by Vision Pro’s hand-tracking.

Made by Marvel and ILM Immersive, the production quality meets the high bar you’d expect from this kind of experience. Though scarcely an hour long, the experience is polished with strong sound, visuals, and voice acting throughout.

Image courtesy Marvel Studios, ILM Immersive

Fittingly for the emphasis on the Marvel multiverse, the narrative plays out across several distinct mediums. Players will see Marvel characters between their own living room (thanks to augmented reality), in fully immersive virtual spaces (thanks to virtual reality), and in 3D animated videos that play on floating screens in front of you.

The jump between these distinct contexts is actually handled very smoothly (aside from an awkward lack of environmental occlusion in AR), whether taking a portal from your living room into a fully immersive space, or seeing a 3D video playing back on a crystal shard floating in front of you (as a representation of some vignette from another timeline).

From a gameplay standpoint, What If…? shows us both the potential and likely pitfalls of fully immersive experiences on Vision Pro.

On the one hand (pun intended), the game makes use of hand-tracking to detect a range of gestures. You’ll use different gestures to summon an arm shield, grab distant objects with a force-like power, or shoot lasers from your hands. Some more complex gesture interactions are used effectively too, like a spell where you pinch your fingers together on both hands then pull them apart to draw a line, rotate your hands to twist the line, then finalize the spell by opening your hands and pushing them together, palm-to-palm.

But even though all of these gestures work reasonably well for this casual experience, they ultimately feel just like that—gestures: simple, prescribed hand motions. You’re never actually touching anything in the experience… just making a gesture and watching it cause things to happen.

I’ve said it before: the least interesting interactions in VR are those that happen far away from you. In VR, it’s not the distant bad guy falling over after you’ve shot them with an arrow that makes VR unique. What’s unique about VR is the fact that you can literally reach over your shoulder to pull out an arrow, nock it to your bow, stretch the string back, aim with one eye, and release—like you had an actual bow in your hands.

Now those reading along carefully should stop to think “isn’t the bow scenario you just described really just a series of ‘gestures’?” Sure. If you want to use the broadest definition of that word, you could dump it into that category. But it’s the fidelity of the gestures that we’re concerned about. It’s the difference between wiggling a Wiimote to make the character on screen swing their sword, versus actually swinging the sword yourself—dictating with precision exactly where it should land on the enemy.

SEE ALSO
VRChat Lays off 30% of Company, Citing Growing Pains Following COVID Platform Boom

There’s no doubt that Vision Pro as a headset is capable of doing the latter. But it has to be said… it’s damn tough without the precision of tracked controllers.

Falling back to hand gestures—and using those hand gestures to cause distant things to happen—isn’t highly engaging. Not being able to reach out and touch the virtual world around you—even to just pick up a rock and throw it with your own hand—diminishes much of the magic of VR.

And that’s where we find What If…?. It’s doing more than I even would have expected with Vision Pro’s hand-tracking. The gestures are varied and detected fairly consistently, but what you do with them feels kind of… detached. I never managed to cross the chasm from I’m ‘doing a hand gesture’ to ‘I’m casting a spell’… and further still from ‘I’m actually interacting with this world’.

Now to be fair, that’s partly due to the experience’s gameplay itself. It’s very casual by the standards of any ‘gamer’; easy enough that most people will be able to figure it out. That’s clearly on purpose; this is a supposed to be a narrative as much as an interactive experience. Marvel and ILM Immersive could have made the gameplay deeper and squeezed a bit more engagement out of the hand-tracking. It’s just hard to see it crossing that chasm I talked about above.

But let’s concede that this casual gameplay is ultimately in service of telling a story. Is the narrative any good? Well, my opinion will surely be colored by the fact that I’m not terribly attuned to the Marvel Cinematic Universe (even if I’ve caught the rough strokes over the years). Dropping me into a What If…? experience—that’s expressly about throwaway alternate timelines—certainly doesn’t lend itself to meaningful character development or real stakes. The narrative has the distinct feeling of being a framework for this experimentation in this new medium, rather than a story unto itself.

So in the end, is What If…? worth checking out on Vision Pro? Sure. I can’t say it was a terrible amount of fun, but hey, it’s free and well produced. And if not for the entertainment value, it’s a clear lesson on what can and probably can’t be done with the headset’s hand-tracking input. In the end this will go down as an early experiment, but certainly not a killer app.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • It should be renamed “What if… they used hand tracking in a better way?”

    • Stephen Bard

      Some of us have already grown bone-weary of valiant resourceful creative experimental attempts to make hand tracking reliable/relevent, with frustrating try-that-gesture-a-third-time half-assed results. You have no alternative on the AVP if you really insist on trying to finagle this device into doing game-like things it was not designed/optimized to do. Hand tracking’s “Golden Age” is Never, except for minimal mouse-clicks. Poking air instead of pulling a trigger ain’t gonna cut it. Thank goodness for my wonderful Quest 3 controllers!

  • polysix

    Trash. Yet another concession to casualising VR down to the point it may as well not exist. Same for standalone games on Quest. Low poly trash with about as much immersion as ant attack 3D (google it).

    The best VR on PC (or PSVR2) can show how utterly amazing VR can be, even at this early stage of tech, then all this casual stuff works against it to tell the world VR is a novelty, a gimmick and frankly.. boring.

    • Stephen Bard

      Yes, trash. But to your next point, having started out as a PC VR guy, I sorta see your perspective, however, I now know for a fact that "many" beefy standalone titles released in the last 8 months and optimized for Quest 3 are actually every bit as good graphically as "most" of the best PCVR titles. To be sure, most titles on both standalone and Steam don't begin to utilize the PC or Quest 3 capabilities and I lament lazy devs in both realms. My favorite Standalone example is Red Matter 2, where the textures and dynamic realtime reflections and shadows look identically impressive to me on Quest 3 and PCVR. Many of my old Oculus friends, who used to hang out together in our really cool Oculus Custom PC Homes (killed unaccountably by Meta), use their New-fangled Quest 3 Only for PCVR because they don't realize how good "some" of the standalone has gotten. Every week I am impressed by how graphically-lame the latest Steam VR offerings look. Yesterday I was watching the Playstation broadcast of upcoming offerings, and wishing that the gorgeous graphics of a few of those flat titles were coming out in VR.

      • "… don't even BEGIN to utilize Quest 3's capabilities …."

        M'friend, you just hit the nail right on the head.
        []^ (