Google Lens is great for when you want to quickly translate a menu written in a foreign language, or visually explore the world around you simply using your Android smartphone. In effort to bring some of those machine learning functions into a VR environment, Twitter user ‘Phasedragon’ recently showed off a new workaround that lets him use Google Lens in VR.
As reported by 9to5 Google, Phasedragon demoed Google Lens in VR by translating a few bits of Korean text from what appears to be a recreation of a Korean train station. Considering however it’s using the full Google Lens suite of tools, we bet a lot more is possible.
To do this, Phasedragon says in a followup tweet that he “just hooked together a few apps,” and tried “a bunch to see which ones worked.”
Phasedragon, also known for tinkering with VRChat on his YouTube channel, says that he initially tried Microsoft Translate to step over some integration issues, but concluded that Microsoft’s version was “simply not as good as Google Translate.”
In the implementation Phasedragon used Sparkocam to capture the desktop and export as a virtual webcam. He then used Android Studio Emulator to run Google Lens, and OVR toolkit to display it in VR.
Although it’s admittedly an impressive bit of software kitbashing, and not anywhere near an official use case, the thought of being able to bring some of the AR functionality of Google Lens into VR is pretty exciting to say the least. Should Google ever invest time into making an official Lens overlay for VR, it could lead to new and exciting types of games, as developers come up with novel ideas of leveraging Google’s machine learning in their creations.