What Are They Working On?
It turns out that Devine gave a presentation in August at Unite 2014, Unity’s developer conference (he says that Magic Leap is using Unity to develop games for the platform). His talk gives almost no useful information (as with the rest of the company’s communications) and left the audience puzzled. He does however mention at the beginning that he was as skeptical as anyone before getting a look at Magic Leap’s technology.
“…I signed the NDA and they showed me a video and I said ‘No, that is impossible. That thing does not… that’s impossible. I know what is possible and impossible—I’ve been at Apple for two years, I helped launched the iPad—that is impossible. I will come to Florida and I will call you charlatans’,” he told the audience at Unite. “…I walk into their office… I stuck my head into what’s called ‘The Beast’—if you’ve ever seen the movie Brainstorm, it’s like the original Brainstorm thing—and, holy cow… it was real. Absolutely incredible. I couldn’t believe it. I made myself a very bad deal and said “I’ll will help you’, within 20 seconds—I couldn’t believe what was possible.”
Development Kits in the Next Year?
The only tidbit mentioned during the talk about the Magic Leap device itself is that it will be wearable (he wouldn’t even specify if it would be head-worn, but that’s probably a safe bet).
Devine also shared a photo of what appears to be a hologram of a shark floating in the middle of the room, he seems to imply that the photo is undoctored, or perhaps it’s a concept of what the company’s technology can supposedly do—create high fidelity, full color, hologram-like imagery that works for augmented reality. The company’s website is filled with similar imagery, including a short video showing a child revealing a tiny elephant in her hands and gasping with amazement.
“There was a guy hanging out in a room, Mick, he’s real, the room’s real. That shark, hanging out in the middle of the table? That’s not meant to be there is it? That’s a real shark—it would be great for shark week. It hangs out in the middle of the table and it’s not meant to be there, what the hell is that? What’s going on?” he teased.
At the end of his talk, one member of the audience asked when development kits of the device would be available. “We’ll be continuing to roll out in the next year,” he said at first, followed by a bit of backtracking, “…but there’s no announcement on dates or anything to do with that.”
Light-fields, Vergence-Accommodation Conflict, and Retina Resolution
In his presentation, Devine mentioned a New York Times article from July which talks almost as vaguely about Magic Leap, but at least confirms that the company is working with light-fields. A light-field display could solve the issue of vergence-accommodation conflict that’s present in current head mounted displays like the Oculus Rift.
Human eyes use two primary means of focusing on objects, vergence and accommodation. Vergence is when both eyes rotate inward to focus at the same point in space; if you cross your eyes, you see double because your eyes don’t have the correct vergence on the objects in front of you. The other means of focus is called accommodation, and it happens per-eye, by bending the lens of the eye to focus light from a certain distance onto the retina; if you close one eye and focus on your finger held close to your face, the objects behind it will become out of focus, and vice versa.
Vergence and accommodation are thought to be linked, because they work in tandem to give us a focused 3D view of our surroundings. Vergence-accommodation conflict can arise with current VR tech where the vergence is variable, but accommodation is static because the light from objects is always being emitted just inches from your eyes rather than bouncing from actual far away objects.
It would seem that Magic Leap’s “Cinematic Reality” technology proposes the use of a head mounted display that uses light-field technology to fix this problem.
Modern attempts at near-eye light-field displays involve using micro-lens arrays which take the light from many tiny images and focuses it into a single image. From my understanding, forming the image in this way allows the vergence and accommodation cues to be in sync because the light from virtually distant objects arrives at the eye as though it is really coming from a distant object.
Nvidia Research demonstrated interesting findings on near-eye light-field displays at SIGGRAPH 2013:
According to the New York Times piece, Abovitz said that the company’s tech is capable of resolution close to the power of the human eye. That’s significant because achieving a pixel-based “retina” resolution (where the eye cannot distinguish individual pixels) would require something like 16,000×16,000 at 100 a 100 degree field of view, according to Michael Abrash, Oculus’ Chief Scientist, something we’re not likely to see in the near future.
