Google Lens is great for when you want to quickly translate a menu written in a foreign language, or visually explore the world around you simply using your Android smartphone. In effort to bring some of those machine learning functions into a VR environment, Twitter user ‘Phasedragon’ recently showed off a new workaround that lets him use Google Lens in VR.

As reported by 9to5 Google, Phasedragon demoed Google Lens in VR by translating a few bits of Korean text from what appears to be a recreation of a Korean train station. Considering however it’s using the full Google Lens suite of tools, we bet a lot more is possible.

https://twitter.com/Phasedragoon/status/1164464933079093249

To do this, Phasedragon says in a followup tweet that he “just hooked together a few apps,” and tried “a bunch to see which ones worked.”

Phasedragon, also known for tinkering with VRChat on his YouTube channel, says that he initially tried Microsoft Translate to step over some integration issues, but concluded that Microsoft’s version was “simply not as good as Google Translate.”

In the implementation Phasedragon used Sparkocam to capture the desktop and export as a virtual webcam. He then used Android Studio Emulator to run Google Lens, and OVR toolkit to display it in VR.

Although it’s admittedly an impressive bit of software kitbashing, and not anywhere near an official use case, the thought of being able to bring some of the AR functionality of Google Lens into VR is pretty exciting to say the least. Should Google ever invest time into making an official Lens overlay for VR, it could lead to new and exciting types of games, as developers come up with novel ideas of leveraging Google’s machine learning in their creations.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 4,000 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.