Google has released a free app for PC VR headsets called Welcome to Light Fields. The company says the app serves as a showcase of “the emerging technology Google is using to power its next generation of VR content.”
When it comes to capturing the real world for VR, 360 photos and videos can only go so far, and have a number of limitations which make them less immersive than computer generated VR content that’s rendered in real-time—namely the missing ability to actually move within a captured scene. Since 360 photo and video content is limited to being seen from only the precise location of the camera, you’re effectively stuck in place, except for being able to rotate your head.
Volumetric capture techniques aim to capture not just one circular perspective, but a totality of the scene (or a at least portion of it), so that viewers can move their heads through 3D space within the capture and see the scene from varying perspectives. Light fields are one promising type of volumetric capture and could represent a flexible, high quality, foundational format for capturing, generating, and storing VR content.
Initially the scenes look similar to simple 360 stereoscopic photos, but the magic happens when you move your head through space—instead of the world being effectively ‘locked to your head’, you’ll see it move around you just like you’d expect if you were really standing there; it’s much more immersive than a static 360 capture.
Google’s custom light field camera takes about one minute for a complete spin to capture a light field scene | Image courtesy Google
It does seem like magic, but it’s not without limitations. The captures are generated in this case by a custom lightfield camera which spins an array of GoPro cameras in a circle to capture a spherical area about two feet wide—you can view the scene from anywhere inside that sphere, but if you stick your head outside of it, the world will go blank. That’s because the cameras effectively capture all of the light rays intersecting the sphere on all sides, and then algorithms recreate the view from any point within the sphere for your viewing pleasure. A larger viewing area can be achieved with a larger camera.
Welcome to Light Fields features some very impressive light field imagery. At its best you’re seeing sharp, immersive views with lots of depth and reflections which convincingly react as you move your head. But there’s some less than stellar scenes (especially if you manually delve into the app’s gallery) that show some of the challenges of capturing quality light fields.
Some of the scenes are not nearly as sharp as the others, and you can sometimes spot artifacts at the edges of objects, mistaken depth information, and reflections and lighting that don’t seem to act quite right. And there’s still the big challenge of filesizes. The handful of static light field scenes in Welcome to Light Fields clock in around 6GB total—fine for a demonstration app, but arguably too large for mass adoption.
Still, Welcome to Light Fields is a powerful example of light field technology and a look at why volumetric capture is probably the future of VR video.