Apple is adding support for Vision Pro’s unique input system to WebXR, the web standard which allows XR experiences to run right from a web browser.

One of the most unique things about Apple Vision Pro is its input system which eschews motion controllers in favor of a ‘look and pinch’ system which combines eye-tracking with a pinch gesture. On the whole it’s a really useful way to navigate the headset, but because it works so differently than motion controllers, it doesn’t play too well with  WebXR.

But Apple is working to fix that. This week the company announced the latest version of VisionOS (1.1) includes a new input mode for Safari’s WebXR capabilities called ‘transient-pointer’. This new mode provides inputs from the headset in a standardized way which developers can use to understand what users are selecting inside of a WebXR session running on Vision Pro.

Up to this point, WebXR apps typically expect a headset report a continuously updated position of each controller. But Apple says it built Vision Pro’s input system to reveal as little information about the user as possible, so it doesn’t report the pose or position of the user’s hands by default. Instead, it only reveals such information at the moment of the user’s pinch (though it’s possible for a WebXR app to ask for full hand tracking info).

 

With the new transient-pointer option, when a user pinches the WebXR app will be able to see a ray representing the direction of the user’s gaze and the coordinate position of their pinch. Like in VisionOS itself, the app thus looks at the pinch to decide ‘when’ a user is making an input, and looks at the ray to decide ‘where’ they’re making the input.

For the duration of the pinch, the position of the pinch itself is continuously updated, allowing for interactions like dragging, pushing, and pulling objects. But when the pinch is released, the app no longer has access to the direction the user is looking or where their hand is located.

SEE ALSO
New Quest 3 Update Improves Passthrough, Brings 'Lying Down Mode' & External Mic Support

With these new capabilities, WebXR apps will be able to adapt their interactions to work correctly with Vision Pro.

However, WebXR on Vision Pro is still experimental. Developers must manually enable WebXR capabilities by accessing advanced settings of Safari in the headset. Developers can also experiment with WebXR and the transient-pointer mode using the VisionOS simulator.

The transient-pointer mode for Vision Pro is being baked into the WebXR standard, and has been added to the most recent draft version of the specification. That means that devices which adopt the same input mode will be able to tap into the same WebXR capabilities.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • STL

    And here, Apple goes the way of PC VR! Not in terms of connectivity, but in terms of „a little software add-on here, a few new parameters there“. Took me 3 whole month of research, reading and experimenting with the literally hundreds of little settings to get modded SkyrimVR running smoothly and in high resolution on PC VR / Quest 3. I wonder how many people would have given up in the process. Will it soon be the same with formerly „plug and play super user friendly“ Apple devices?

  • Good that at least they are contributing to one standard…

  • ViRGiN

    Apple: adds new real feautres
    Valve: bueheheheh, spring sales!11!