Devs Use ARKit to Make an iPhone Motion Controller for HoloLens

News Bits

5

Ok, these ARKit experiments and mashups are getting darn cool. Here VR development studio Emergent linked together HoloLens and an iPhone with ARKit to turn the iPhone into a 6DOF motion tracked controller.

The tracking may be somewhat rudimentary, but it’s becoming increasingly clear the many possibilities that come with smartphones that are able to understand their position in the world. From a fusion of AR and VR to a portal into volumetric video, we’ve seen some really cool ARKit experiments, but none that have used an iPhone as a motion controller instead of a display.

Emergent has demonstrated the possibility by turning an iPhone into a motion controller for a HoloLens game. In the game the HoloLens is represented as the player’s helmet, while the iPhone is a sci-fi laser gun that’s tracked in all dimensions. Taking things a step further, the studio demonstrates how bullet laser holes can be convincingly overlaid onto objects in the environment, thanks to the devices’ ability to map the geometry of the environment.

It’s clear from the video that there’s a good deal of latency and that the precision isn’t particularly high (after all, ARKit isn’t intended for immersive experiences), and keeping the tracking systems of both HoloLens and ARKit in sync over distance and time could post a challenge. But the potential for the use-case is clear—especially as smartphones begin to hit the mainstream with more precise motion tracking instruments—a well tracked smartphone can be a flexible and widespread tool for AR and VR alike.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • CoffeeBuzz

    Multiplayer AR gaming would be fun BUT I dont see this happening for years. You would have to be in the same real/playspace to get the same environment scans.

    • NooYawker

      AR is much more complicated than VR. I mean every game is already 360, many already can run at 90fps. AR on the other hand has to look real against the actual environment. You’re right, it’s years away from delivering a viable experience.

    • dk

      yep a couple of years ….in the meantime search for hololens and sort by date there r a ton of developers uploading content constantly……and with future headsets u wouldn’t have to choose between the 2

  • Kris Bunch

    Waving around weapon like things, running full speed, taking cover… sounds like a good way to get shot in RL by either law enforcement or a well meaning armed citizen.

  • Very nice experiment