Niantic, the developer behind Pokémon Go and the upcoming Harry Potter AR game, announced they’ll be opening up access to their latest work in AR, dubbed the ‘Niantic Real World Platform’, to third-party developers soon. To boot, the company also introduced a few key AR technologies that will have you salivating over the possibilities of actually chasing down pocket monsters on your commute to that next Pokéstop.

Niantic says their Real World Platform blends machine learning and computer vision to tackle the classic challenge of building a useful and realistic AR experience on mobile devices—something that can sense small details, understand surroundings, and model them in an interactive 3D space that a smartphone can digest.

One area of research Niantic has been work on is proper occlusion, or making sure digital imagery fits into the physical world correctly, and allowing it to be obscured naturally by objects in the environment. The company published a quick video on their blog, showing off their latest work in the area of AR occlusion. What better test subject than Pikachu?

Creating correct occlusion in AR requires that the computer, in this case a smartphone, contextually understands the world around it. Slowing down the video some, it becomes a little more clear however that the company still has a ways to go, as the occlusion masks oftentimes overcompensate, or misjudge the alignment of objects as Pikachu scampers about. While a proof-of-concept, it’s definitely a tantalizing look at the near future of smartphone AR, and a clear departure from what we saw at Pokémon Go’s launch back in Summer 2016.

 

The Niantic Real World Platform is also focusing on cross-platform AR for shared, multiplayer experiences. The biggest obstacle, the company says, is invariably latency. To this effect, the company says they’ve developed “proprietary, low-latency AR networking techniques” to overcome this problem, which allowed them to realize a unified, cross-platform solution with a single code base. To demonstrate, Niantic built a multiplayer smartphone AR shooter, dubbed ‘Neon’, which shows six users playing at once.

We can attribute some of this to the company’s recent acquisitions; Niantic recently acquired Escher Reality, a studio touted for its cross-platform, multi-user AR platform, and the computer vision and machine learning company Matrix Mill—two decisive moves forward after the company’s $200 million Series B finance round.

SEE ALSO
Vision Pro Ordering Process Will Use Face ID Scans to Determine Custom Fit

“It’s through the coordination of these teams that we’ve been able to establish what the Niantic Real World Platform looks like today, and what it will be in the future,” Niantic says in a statement.

As for third-party developers looking to get in on Niantic’s platform, the company says they’ll be picking a “handful” of devs to begin working with their tools later this year. To receive more information, sign up here.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Scott Hayden
Well before the first modern XR products hit the market, Scott recognized the potential of the technology and set out to understand and document its growth. He has been professionally reporting on the space for nearly a decade as Editor at Road to VR, authoring more than 3,500 articles on the topic. Scott brings that seasoned insight to his reporting from major industry events across the globe.
  • Ian Shook

    The slight occlusion errors aren’t nearly as distracting as the basic tracking errors.

    • Lucidfeuer

      These slight errors are what make me confident in their development: it looks and and acts real, and those artefacts show this is already possible and running.

      Now the question is whether they’re really just using a single smartphone unit and camera…

      • David Mulder

        Exactly, I have seen countless perfect demos… and knowing what is and isn’t possible with modern day technology I just dismiss those out of hand. This however looks pretty real, so it got me at least a tiny bit excited (Though I personally thing AR is a weaker version of VR for most applications).

  • Jay degaris

    How about the FUNDAMENTAL issue of looking at your pathetic mobile device to ‘play’ these boring ideas. AR is Great but not viewing from the phone screen, it’s so childish and pathetic.

    • Engineer_92

      You realize that’s the goal right? Tech is moving fast but these things do take time.

      • Jay degaris

        Yes that is the grand holy goal, wow you sound so naive! They just acquired Escher Reality after a 200 Million dollar funding (by convincing gold digging utterly dumb investors on the viability of these ass holes to make money on the AR fad).

        Dont think they’re just going to give their tech away willy nilly, it’s ALL part of their deep long term marketing ploy, as is the case for MOST companies in the current VR space.

        • Engineer_92

          Lmao, you’re calling me naïve? Look up sarcasm!! You complained about the fundamental issue of having to look through a smartphone. Obviously that isn’t the long term goal for AR. Ive done my research and I understand the marketing ploy. I was only responding to the ridiculous content of your first post.

          • Jay degaris

            Dont worry you will understand what I mean in 10 years. You should do a little more research. You’re still too young unfortunately.

          • Engineer_92

            The proverbial “You’re too young to understand argument”. That’s all you got? Just goes to show that you were spouting off an unnecessary comment. I knew exactly what you were trying to say, you just couldn’t comprehend the sarcasm in my response. And that’s ok

          • JJ

            yeah jay must be high, or have no grasp on the fact that technology takes time to develop and isn’t instant. Of course none of want to look into our phone to see AR outside of the phone, but this is where we are in technology and we have to get down basic image reco and depth sensing before we can do anything else. So its great that we can iron these features out in a contained environment so that by the time more AR headsets come out these are all concrete and work as expected.

    • ra51

      Guess jay forgot about a little thing called Pokémon Go from a couple years ago that took the world by storm.

  • Firestorm185

    Can’t wait until a big player makes an MR headset with VR and AR and we get a version of Go with solid 90fps and good world tracking so you can actually have a buddy follow you in AR.

  • WyrdestGeek

    I lack confidence in Niantic’s ability to do any of this given how thoroughly they kept dropping the ball in development of Pokemon Go, and especially the debacle in Chicago.

    No excuse. They should have been aware that, that many people close together would overload cell towers. They should have planned with WiFi.

  • DC

    Very nice, but still a ways off from believable AR in a 360 AR headset.

  • RJH

    I like this. Looks great.

  • Muzufuzo

    AR development is going very slowly, both in hardware and software. I remember playing with AR back in 2010.