Chris Madsen steps into the beloved universes of Star Wars and Jurassic World via ILMxLab’s ingenious 3D C.A.V.E experience which uses positional tracking to create the illusion that you’re inside the film. It’s called ‘Holo-Cinema’.
While at the Sundance Film Festival, I ran into Rob Bredow, Head of New Media and VP of Advanced Development Group. Rob told me that Lucasfilm had been invited by Sundance to show off their augmented reality multi-user stage they have dubbed Holo-Cinema in which multiple users can step together into a shared virtual environment with no equipment other than a set of lightweight glasses. This type of setup is often referred to as a CAVE system (Cave Automatic Virtual Environment) commonly associated with university VR labs. Not only had I been wanting to try a system like this for years, I was now being invited to be transported to the surface of Jakku from the latest Star Wars film and play with a raptor in Jurassic World. I immediately cleared my schedule.
The positional data is used to determine how the images are projected relative to the user, making the seams where the walls and floor meet match up appropriately. The effect is impressive even without the 3d glasses (see video at :30 when C-3PO appears to step out of the wall keeping in mind that this was shot with the monoscopic lens of my Samsung Note 4 camera). With the glasses on, the environment becomes full of 3 dimensional ‘holograms’ that can be interacted with and give the illusion that one could just keep walking off into the distance (which one woman apparently tried to do during a demo).
As someone who uses VR nearly daily, it felt incredible to walk, run, kneel and lean freely around the virtual objects without wires or burdensome hardware. The glasses were easily passed back and forth between the users. Matt Terndrup from Two Bit Circus pushed the Holo-Cinema to its limits by literally rolling around on the floor and under a piece of virtual junk that was littering the desert. He did break it for a few seconds when the glasses were occluded from the cameras but was able to talk about the decal that was apparently stuck to the bottom side of the virtual object.
Holo-Cinema has been in operation in various forms as an experimental lab piece for years now at ILMxLab. Hilmar Koch, Head of Virtual Production, credits the legendary John Gaeta, Executive Creative Director and the man behind the award winning visual effects of The Matrix for bringing the technology to the lab. According to Koch, John declared; “We are going to build this because it’s going to change things”. Turns out he was right. Not only is the Holo-Cinema an incredible visualization tool, it’s also highly effective in project collaboration. Moving beyond the ‘green screen’ the Holo-Cinema allows multiple creatives such as writers, sound developers, immersive ride developers, game designers, AR/VR developers and others to be simultaneously immersed into the scene to discuss, point at and manipulate the environment in real time.
What we saw at Sundance were short simple vignettes. Koch says they want to take this experience to the next level and make it truly interactive. They have proven they can run this with a live C3PO that will respond to you, react to your movements, look you in the eye, and more. While this level of interaction was not on display today, I did get a taste of what it might be like. When approached by the raptor, I began stroking his head when it suddenly nipped at my fingers. I withdrew my hand, and it appeared to lunge at it multiple times. The team envisions interactive group cinematic experiences, live theater in which you are a participant, education and more.
Hilmar laughed at my experience with the dinosaur, telling me I had experienced the genius of the actor behind the dinosaur performance who he claims is so gifted he can predict the behavior of what the majority of people might do in any particular encounter and preemptively act out the creatures responses.. Having fallen for it, I can attest the performance really does give the illusion of a living, thinking creature telling me to beware and keep my hands to myself.
The enthusiastic response from the Sundance crowd has been inspiring to the team and an indicator they are on the right path for the out-of-home entertainment market. To be able to spend an evening on a Holo-Cinema stage with friends, interacting within an environment that is aware of your presence sounds like something out of science fiction but in reality is just around the corner. I’m ready.