Developers of mixed reality games have been clamoring for passthrough camera access on Quest for a while now, something now-departed Meta VP of XR Mark Rabkin said would come with the release of a new camera Passthrough API sometime in early 2025. Now, Google has confirmed that Android XR includes access to passthrough cameras, letting developers fine tune their mixed reality experiences from the get-go.

As reported by XR developer Antony Vitillo of Skarred Ghost, Google has confirmed that Android XR app developers can request permission to access both passthrough and face-tracking cameras, the latter of which only provides an “avatar video stream,” ostensibly allowing a degree of user anonymity.

Meta Quest 3S | Photo by Road to VR

On Quest and Vision Pro, third-party developers only have access to spatial data and the ability to use passthrough as a background, but not the ability to directly interact with captured frames—making it difficult to apply AI and machine learning algorithms to better understand the user’s space. The only exception right now is enterprise apps on Vision Pro.

On the other hand, Google confirmed Android XR app devs can make use of its various APIs, including Camera2 and CameraX. This critically does allow real-time frame analysis, which can feed frames directly to a machine learning model to better understand a user’s surroundings.

SEE ALSO
Somnium Begins Shipping VR1 PC VR Headset in the US, Announces Incoming Price Hike

As Vitillo notes, Android XR is still in preview and the long-awaited first device running it, Samsung Project Moohan, hasn’t been officially released. At this point, it’s anyone’s guess what will make it through to the first public release, which is ostensibly planned for consumer launch sometime this year.

More broadly, this essentially means Google is treating camera access on XR headsets similar to how it handles those permission on smartphones, with the move more closely aligning Android XR with the broader ecosystem—which notably also includes app compatibility across XR and mobile and those all-important camera consent prompts.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Scott Hayden
Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Ivan

    It might also be interesting to add to your article that there are other existing AR headsets that already provide camera access in addition to Apple's Vision Pro. For example, standalone AR glasses like the Pico 4 Enterprise, HoloLens 1 and 2, Magic Leap 1 and 2, several Vuzix glasses, etc.

    If I'm not mistaken, most recent PCVR headsets (2013 and beyond) allow camera access, either by using the on-board camera or by attaching an external camera (either to the headset or to the computer).

    Technically, Quest developers can currently get camera access for image processing/AR use, but they have to use an external camera using Meta's Ocean framework.

    Hopefully, Meta will take a cue from Google's camera access method and require confirmation of camera access from the user. Companies could also add a constant visual cue in the UI to indicate that the camera is currently being accessed (similar to when you record your screen). While a constant camera access cue may be annoying and not look good, it will probably give users some peace of mind that they can easily tell if the camera is being accessed or not. Hopefully Meta will release the camera API soon, otherwise they will fall even further behind the competition.

    I'm curious to see if Google or Meta will require developers to specify in their app description what type of access is required (camera, microphone, etc.) and also curate these apps, or at least test to make sure nothing is being done wrong with camera access, before allowing an app to be published on their stores. This would be a good way for users to know that an app requesting camera access is safe because it has been vetted.

    I also hope that developers will not do something wrong with camera access and ruin it for other developers.

    • Arno van Wingerde

      Your "hope" is my "fear": how does allowing developers (i.e. everybody allowing a view of your room, people you meet etc. not translate to an even more brutal rape of the users' privacy? Of course, these are people dealing with meta already, so everybody (including myself) here has already given up their privacy when they started WhatsApp. Actually even earlier than that: since their friends started using WhatsApp/Facebook/…
      Of course, with the current US "government", I know my data are in safe hands…

  • Thanks for the mention, Scott!

  • David Cano

    No, is an important tool for Meta Apps to steal even more personal data. Vision Pro demonstrated that there's no need to send the Apps the exact images the cameras are seeing, only the surfaces, so no App can steal your privacy.