Stepping Into Star Wars and Jurassic World with ILMxLab’s ‘Holo-Cinema’

4149
10

Chris Madsen steps into the beloved universes of Star Wars and Jurassic World via ILMxLab’s ingenious 3D C.A.V.E experience which uses positional tracking to create the illusion that you’re inside the film. It’s called ‘Holo-Cinema’.

While at the Sundance Film Festival, I ran into Rob Bredow, Head of New Media and VP of Advanced Development Group. Rob told me that Lucasfilm had been invited by Sundance to show off their augmented reality multi-user stage they have dubbed Holo-Cinema in which multiple users can step together into a shared virtual environment with no equipment other than a set of lightweight glasses. This type of setup is often referred to as a CAVE system (Cave Automatic Virtual Environment) commonly associated with university VR labs. Not only had I been wanting to try a system like this for years, I was now being invited to be transported to the surface of Jakku from the latest Star Wars film and play with a raptor in Jurassic World. I immediately cleared my schedule.

ILMxLabs-Holo-Cinema-1The ILMxLAB’s Holo-Cinema contains a pair of walls that run the perimeter of two sides of the large square floor. Powerful projectors beam high definition 3D stereoscopic images onto the surface of the walls and floor. Small, lightweight reflective balls are mounted to the 3d glasses and are used in conjunction with special cameras pointed towards the stage that determine exactly where and what angles the user’s head is at any given time in the real world environment.

SEE ALSO
Watch Artist Wesley Allsbrook Create 'Live' in VR with Oculus' 'Quill'

The positional data is used to determine how the images are projected relative to the user, making the seams where the walls and floor meet match up appropriately. The effect is impressive even without the 3d glasses (see video at :30 when C-3PO appears to step out of the wall keeping in mind that this was shot with the monoscopic lens of my Samsung Note 4 camera). With the glasses on, the environment becomes full of 3 dimensional ‘holograms’ that can be interacted with and give the illusion that one could just keep walking off into the distance (which one woman apparently tried to do during a demo).

As someone who uses VR nearly daily, it felt incredible to walk, run, kneel and lean freely around the virtual objects without wires or burdensome hardware. The glasses were easily passed back and forth between the users. Matt Terndrup from Two Bit Circus pushed the Holo-Cinema to its limits by literally rolling around on the floor and under a piece of virtual junk that was littering the desert. He did break it for a few seconds when the glasses were occluded from the cameras but was able to talk about the decal that was apparently stuck to the bottom side of the virtual object.

Holo-Cinema has been in operation in various forms as an experimental lab piece for years now at ILMxLab. Hilmar Koch, Head of Virtual Production, credits the legendary John Gaeta, Executive Creative Director and the man behind the award winning visual effects of The Matrix for bringing the technology to the lab. According to Koch, John declared; “We are going to build this because it’s going to change things”. Turns out he was right. Not only is the Holo-Cinema an incredible visualization tool, it’s also highly effective in project collaboration. Moving beyond the ‘green screen’ the Holo-Cinema allows multiple creatives such as writers, sound developers, immersive ride developers, game designers, AR/VR developers and others to be simultaneously immersed into the scene to discuss, point at and manipulate the environment in real time.

SEE ALSO
'Star Wars: Trials on Tatooine' Coming to HTC Vive, Leaked Trailer Indicates

What we saw at Sundance were short simple vignettes. Koch says they want to take this experience to the next level and make it truly interactive. They have proven they can run this with a live C3PO that will respond to you, react to your movements, look you in the eye, and more. While this level of interaction was not on display today, I did get a taste of what it might be like. When approached by the raptor, I began stroking his head when it suddenly nipped at my fingers. I withdrew my hand, and it appeared to lunge at it multiple times. The team envisions interactive group cinematic experiences, live theater in which you are a participant, education and more.

Hilmar laughed at my experience with the dinosaur, telling me I had experienced the genius of the actor behind the dinosaur performance who he claims is so gifted he can predict the behavior of what the majority of people might do in any particular encounter and preemptively act out the creatures responses.. Having fallen for it, I can attest the performance really does give the illusion of a living, thinking creature telling me to beware and keep my hands to myself.

The enthusiastic response from the Sundance crowd has been inspiring to the team and an indicator they are on the right path for the out-of-home entertainment market. To be able to spend an evening on a Holo-Cinema stage with friends, interacting within an environment that is aware of your presence sounds like something out of science fiction but in reality is just around the corner. I’m ready.

  • brandon9271

    I’m a bit confused how this could be a multi-user experience when only one person can wear the glasses are a time. Everything is render from the perspective of that one person. To anyone else it would look distorted.

    • Matthew Hermans

      Possibly the same back-end software could drive through a Holo-Lens style front-end; which is multi-user.

      • brandon9271

        Anything is possible.. We live in the future! lol

    • Duilio Afazzoni

      https://www.youtube.com/watch?v=0PInocdo9YA
      it works for the guy with glasses and the guy with the cam
      I think it works with the shutter glasses the same way the official ps3 3d monitor (and some tvs) handles the multiplayer games at full screen on one monitor.
      The cpu renders two scenes and the projectors project the two scenes with each other frames intervailed.
      The shutter glasses of guy 1 cut out the frames of the scene intended for guy two and vice versa, so guy 1 can only see the scene rendered for guy 1 (even frames) and guy 2 see the scene rendered for guy 2 (odd frames)

      • brandon9271

        Well, if that’s the case it would actually render four scenes because in order to be in 3D it’s two stereo pair per person.. I suppose a 120hz projector could do 30hz per eye, per person which would still be good enough i suppose. It could also do 4 non stereo views and maybe the motion parallax alone would be convincing enough. Hmm.. I’d love to try it out! :)

      • Lhorkan

        If this were the case, you would see both images overlapping on the filmed projection, because even if you were to put shutter glasses in front of a camera, the frame rate wouldn’t match up. If you look at the second video in the post, you can clearly see the two overlayed videos for the sterero vision (still just a single point of view however).

        The only explanation is that the video was staged – only the guy filming is being tracked, and one set of the stereo projection is disabled so you can see a clear image in the video. The guy being filmed is just trying to make it look like he also sees proper visuals, whereas he actually sees a very distorted image.

        • Duilio Afazzoni

          maybe because it was a “single player run”
          with a projector running a 240hz tecnically you can handle 2 different scene in 3d at 60 fps each

  • vaughn whisker

    true but the distortion really isnt that bad if you’re close to the principal user.

  • Christian J Murray

    Good, early-days work, 5 years to commercialize maybe sooner if they bring in some help.