Augmented reality is conceptually so broad, there’s almost no limit to what you could imagine doing with the technology. But rather than throwing out the playbook entirely, we’ll to our grand visions through useful incremental steps that begin with integration into existing content built around today’s paradigms. A concept video using HoloLens and Final Fantasy XIV, shows how AR could be incredibly useful as a ‘Second Screen’ companion for gaming and more.

While one day we’ll want AR games that are entirely based in augmented reality, today’s reality is that much of the world already has high-end, high resolution displays in their home, and powerful computers or consoles already hooked up to them to play all manner of games and content. Making use of those screens—especially while AR display technology and content design is still advancing—could be a great stopgap to a fully augmented futured.

The ‘Second Screen’ idea—putting additional content on a mobile device that’s relevant to what’s playing on the main screen—has been around since smartphone usage has become mainstream in the last decade. We’ve seen it attempted with games and film/TV content in the last several years, but it hasn’t quite caught on as something truly useful. AR, however, could change that.

Take this concept from Twitter user Tanufuku, who paired Microsoft’s HoloLens AR headset with the complex UI of Final Fantasy XIV.

Tanufuku has reserved the laptop screen entirely for real-time 3D action, while exploding the game’s dizzying array of interface windows to the surrounding area with AR via HoloLens.

Now one might ask ‘Well then, why not just play the game on an AR screen too?’ Great question. The reason is that AR display technology is still advancing, and the laptop’s screen is almost certainly the superior display option for the time being.

For instance, while the video makes it seem like you’d be able to see all the AR windows in your periphery while looking at the main screen, the actual HoloLens field of view is around 35 degrees diagonally (making its available viewing area much smaller than most static displays that people use in their home today) which means if you replaced the laptop screen with an AR screen, you’d get clipping if you turned your head even slightly (and the AR screen would disappear completely from your field of view if you went to look at the surrounding AR windows).

SEE ALSO
Lumus Packed a 55 Degree Field of View into Optics Less than 2mm Thick

Keeping the most important parts of the game on a real screen means the less important info (occasionally-access parts of the interface) will clip out of the field of view while you focus on the most important part of the game, but when you go to look at the AR windows, you’ll still be able to see that important real-time game action on the screen in your peripheral view.

microsoft-hololens
Microsoft’s HoloLens AR headset

Not to mention the fact that many users will have monitors or TV’s that far exceed the effective resolution of AR displays for some time, and, if you are running a contemporary game, there’s no reason to use a transparent display (which would degrade the quality of the content due to interference with the real world in the background).

So, leaving the most important part of the content (the real-time action) on a real screen—since they’re inexpensive, high-resolution, and we already own them!—while giving the less important parts of the game (like maps, inventory, menus, scoreboards, and more) some breathing room via augmented reality makes a lot sense for AR’s use as a ‘Second Screen’. Especially with some smart transitions between the real screen and the augmented space; you can imagine how, with some work, a game could allow users to seamlessly drag windows ‘out of their laptop’ and into augmented reality.

Alternative Text

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • dotsmada

    Honestly, I just wish games would allow you to utilize multiple screens at once. I’ve got 4 screens hooked to my computer. It would be great, for example, to have the main game on my main screen, my bag with all it’s content on a second screen, and then a map on a 3rd screen. I could move my mouse between the 3 screens whenever I want and I’d still be in the game.

    • Hernani Pawlick David

      Grid Autosport does something like that.

  • VirtualBro

    Did he say how he did that? FF14 doesn’t support that stuff natively, does it?

    Man, it’d be great just to split the game into two or more regular monitors if nothing else, like Mr. Mada said

  • Gaming aside, I was just lamenting this morning about how i’m suddenly using two tablets, a smartphone, a laptop, and an 22 inch tv monitor just to facilitate my morning routine. Had someone told me I’d be doing that 30 years ago I’d have laughed my ass off in his face. At this point, I would gladly welcome a single HMD to eliminate all this extra hardware…

    • DaKangaroo

      It is the next logical progression isn’t it? It reminds me of those MTV cribs where they have a TV in every room, multiple in some rooms, just so they can always see one. At that point it begs the question, ‘Why not just mount one to your face? It’d be cheaper!’.

      Seems logical to me that some kind of AR/VR computer smartglasses thing will eventually become a mainstream essential component of modern life.

  • OgreTactics

    Well that’s the fucking point of VR. And since glasses format won’t come until 10-15 years, I’m really tired of waiting for manufacturer to sell real practical headsets.

  • NooYawker

    Kind of a shame that he’s playing off a small screened laptop.

  • JoeD

    Yeah, now all you need is $4000 to try it out. Man oh man! I hope it’s as good as the Kinect!!!