What Are They Working On?

It turns out that Devine gave a presentation in August at Unite 2014, Unity’s developer conference (he says that Magic Leap is using Unity to develop games for the platform). His talk gives almost no useful information (as with the rest of the company’s communications) and left the audience puzzled. He does however mention at the beginning that he was as skeptical as anyone before getting a look at Magic Leap’s technology.

“…I signed the NDA and they showed me a video and I said ‘No, that is impossible. That thing does not… that’s impossible. I know what is possible and impossible—I’ve been at Apple for two years, I helped launched the iPad—that is impossible. I will come to Florida and I will call you charlatans’,” he told the audience at Unite. “…I walk into their office… I stuck my head into what’s called ‘The Beast’—if you’ve ever seen the movie Brainstorm, it’s like the original Brainstorm thing—and, holy cow… it was real. Absolutely incredible. I couldn’t believe it. I made myself a very bad deal and said “I’ll will help you’, within 20 seconds—I couldn’t believe what was possible.”

Development Kits in the Next Year?

The only tidbit mentioned during the talk about the Magic Leap device itself is that it will be wearable (he wouldn’t even specify if it would be head-worn, but that’s probably a safe bet).

Devine also shared a photo of what appears to be a hologram of a shark floating in the middle of the room, he seems to imply that the photo is undoctored, or perhaps it’s a concept of what the company’s technology can supposedly do—create high fidelity, full color, hologram-like imagery that works for augmented reality. The company’s website is filled with similar imagery, including a short video showing a child revealing a tiny elephant in her hands and gasping with amazement.

Existing VR Games Would Look Great on Vision Pro, But Without Controllers Most Are Stuck

magic leap light field display

“There was a guy hanging out in a room, Mick, he’s real, the room’s real. That shark, hanging out in the middle of the table? That’s not meant to be there is it? That’s a real shark—it would be great for shark week. It hangs out in the middle of the table and it’s not meant to be there, what the hell is that? What’s going on?” he teased.

At the end of his talk, one member of the audience asked when development kits of the device would be available. “We’ll be continuing to roll out in the next year,” he said at first, followed by a bit of backtracking, “…but there’s no announcement on dates or anything to do with that.”

Light-fields, Vergence-Accommodation Conflict, and Retina Resolution

In his presentation, Devine mentioned a New York Times article from July which talks almost as vaguely about Magic Leap, but at least confirms that the company is working with light-fields. A light-field display could solve the issue of vergence-accommodation conflict that’s present in current head mounted displays like the Oculus Rift.

Human eyes use two primary means of focusing on objects, vergence and accommodation. Vergence is when both eyes rotate inward to focus at the same point in space; if you cross your eyes, you see double because your eyes don’t have the correct vergence on the objects in front of you. The other means of focus is called accommodation, and it happens per-eye, by bending the lens of the eye to focus light from a certain distance onto the retina; if you close one eye and focus on your finger held close to your face, the objects behind it will become out of focus, and vice versa.

'Alien: Rogue Incursion' Finally Announced From Veteran VR Studio, Set for Late 2024

Vergence and accommodation are thought to be linked, because they work in tandem to give us a focused 3D view of our surroundings. Vergence-accommodation conflict can arise with current VR tech where the vergence is variable, but accommodation is static because the light from objects is always being emitted just inches from your eyes rather than bouncing from actual far away objects.

It would seem that Magic Leap’s “Cinematic Reality” technology proposes the use of a head mounted display that uses light-field technology to fix this problem.

Modern attempts at near-eye light-field displays involve using micro-lens arrays which take the light from many tiny images and focuses it into a single image. From my understanding, forming the image in this way allows the vergence and accommodation cues to be in sync because the light from virtually distant objects arrives at the eye as though it is really coming from a distant object.

Nvidia Research demonstrated interesting findings on near-eye light-field displays at SIGGRAPH 2013:

According to the New York Times piece, Abovitz said that the company’s tech is capable of resolution close to the power of the human eye. That’s significant because achieving a pixel-based “retina” resolution (where the eye cannot distinguish individual pixels) would require something like 16,000×16,000 at 100 a 100 degree field of view, according to Michael Abrash, Oculus’ Chief Scientist, something we’re not likely to see in the near future.

Continue Reading on Page 3…

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • I’ve been trying to dig up information all day about Magic Leap… Should have checked here first. The potential of what they’re talking about sounds incredible. After reading the patent descriptions you posted above I’m convinced it’s going to be something truly spectacular. It would have to be pretty convincing to attract a 500m investment from Google.

    Thanks for posting!

    • Ben Lang

      Don’t forget, we don’t have any confirmation or corroboration of the claim that there’s a $500m round on the table, just one unsourced report. Though it would be awesome if their tech is as great as they are making it out to be.

  • elecman

    From NY Times (July 18, 2014):
    “…before creating Magic Leap in 2010, said that his system would even offer a resolution close to the power of the human eye”.

    I hope the tech can generate a FOV of 100+ degrees.

  • Joe Nickence

    Without anyone showing me anything, I’m not about to get all hyped up over this. I’m at the point where anything AR or immersively VR related, will be nothing but Snake Oil, as you said. “We’re waiting until it’s perfect before we release it to the public” simply sounds like “we have cool hi tech toys and you can’t play with them”.

    • cly3d

      You know your stuff Joe! :)

  • cly3d

    I’m skeptical. A lot of “Devine’s” lines sound like tele-evangelism. Yet of-course, something is surely in there if Google is looking at it.

    Meanwhile..>> “…Among the company’s 36 trademark applications are many references to comic books, specifically relating to what sounds like augmented reality functionality. “Comic books enhanced with specialized covers; graphic novels enhanced with specialized covers,” reads one of the applications.

    “Imagine if the imagination could imagine” is probably the zaniest among the trademark applications, along with the company’s oft-used phrases, “Cinematic Reality,” “A Rocketship for the Mind,” and what sounds like an app store for their hardware called “The Magic Shop.”

    Oh c’mon.. tell me this doesn’t sound like tele-evangelism and hyperbole!
    A patent for Graphic novels with specialized covers? if it’s “Magic Book” AR, then it’s been baked a long time ago.

    Hopefully, it’s something radically different and of a higher fidelity than Vuzix’s that you guys covered back in 2012? https://www.roadtovr.com/vuzix-smart-glasses-augmented-reality-glasses/

  • dvoshart

    After briefly skimming the patents it seems like it is a wide FOV google glass. However: the image projected to the eye is a light field image.

    This is awesome but the problems I see are:

    1: Computing power required to do light fields are pretty expensive. Hopefully they use the information from the z-buffer to create the light field. Even if this was the case the eyeball moves left and right you would pick up on the occusion artifacts.

    2.The next problem is that no device has been able to give sub 20ms latency for capturing scene information.

    3.And it appears that everything is additive. You can’t therefore black something out. This limit’s game design.

    • eturner

      Good points. But take another look at the patents and you’ll see they have a solution for #3 — one of their HMD patents describes a “mask LCD / mask display” that can selectively block out incoming light at specific X/Y offsets — this allows them to apply opacity against portions..or all…of the outside world

  • Noam

    It’s really funny reading todays headlines over the internet claiming that Google invests in “Magic Leap” to go head to head against Facebook’s “Oculus Rift”.

    This is bullshit.

    The Magic Leap solution is for augmented reality – they are using light field screen to be able to project 3D images on top of reality. This is something Google needs *badly* for their Google Glass to be a sucess (currently Google Glass can not project 3D images on top of reality, and this makes it a very low quality device for augmented reality).

    The novelty of Magic Leap is not using light field screens, but solving the low spatial resolution, that was an inherent problem for any light field screen created to date.

    To sum this up, Magic Leap is all about being able to do augmented reallity properly, which is what Google needs for their Glasses, it has nothing to do with complete virtual reality devices like the oculus rift, which do not try to augment reality in any way, rather re-create it entirely.

    • Snarl

      that’s not really correct … when they are able to mask the screen in real-time ( how awesome is that !! ) to block outside light.. you have a 100% rift replacement …(except that it is lighter and thinner and most probably more ) .. so a perfect VR AND AR device …