nintendo-eye-tracking

Nintendo has always danced to a tune all its own. Refusing to accept the status quo in the often relentless inevitability of the gaming world has lead to both incredible successes (introducing natural motion with the Wii) to abject disasters (the Virtual Boy), in either case though the Japanese gaming giant usually intrigues with its innovation.

Now, a patent just published (albeit filed in March) in the US indicates Nintendo’s possible targets for pushing immersive boundaries in gaming. The patent details a system which utilises glasses and a tracking camera, mounted above the gaming display which aims uses gaze tracking to alter the image presented to the user.The system seems to suggest tracking based both on a users head position and their gaze direction, with the rendered view altering in relation to those positions to give the illusion for the user of 3D depth. More than that though, the patent details interactive benefits and enhancements such as dodging virtual projectile travelling towards the user in 3D space by merely moving one’s head.

The technology and its desired effect seems to share strong commonalities with the now famous experimental videos posted by Johnny Lee which, ironically, used a reverse Wii-mote and sensor bar system to render a 2D view relative to the user’s head position, adjusting the viewing pane to give an illusion of 3D depth. The videos probably give a good idea of the effect Nintendo desires, albeit enhanced further by tracking gaze as well.

The patent’s extensive background detail also gives examples of immersive enhancements that could join this eye-tracking technology to enhance a player’s connection to the game world. Clearly Nintendo recognise that relying on merely static 2D displays, as featured in the Nintendo Wii-U or even the autostereocopy found in the company’s handheld 3DS will cut it for gamers in the near future.

SEE ALSO
New Vision Pro Hands-on Reports Drop Ahead of Friday Pre-order

Nintendo has commented recently on its interest (or lack thereof) in virtual reality, citing the technology as interesting but too solitary at present. Although Nintendo’s famous Shigeru Miyamoto did make his way over to Oculus VR’s stand to try out the DK2 at the E3 Expo recently.

Patents sometimes give insight into a company’s technical direction, but more often than not it’s merely the sign of them exploring all avenues they can to find something they feel works for them and their audience. But Nintendo would be foolish to dismiss virtual reality entirely as a viable avenue for them. And personally, a VR Zelda with camera mechanics similar to Lucky’s Take, the 3D platformer published by Oculus VR, sounds pretty damned awesome!

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Based in the UK, Paul has been immersed in interactive entertainment for the best part of 27 years and has followed advances in gaming with a passionate fervour. His obsession with graphical fidelity over the years has had him branded a ‘graphics whore’ (which he views as the highest compliment) more than once and he holds a particular candle for the dream of the ultimate immersive gaming experience. Having followed and been disappointed by the original VR explosion of the 90s, he then founded RiftVR.com to follow the new and exciting prospect of the rebirth of VR in products like the Oculus Rift. Paul joined forces with Ben to help build the new Road to VR in preparation for what he sees as VR’s coming of age over the next few years.
  • snake0

    Innovation my ass, they’re just stealing ideas that Wii hackers came up with 8 years ago. The writing is on the wall with regards to 3D. They would be stupid to keep walking down this road. The Wii U was a pitiful disaster too. They should just go 3rd party and then they could make their gimmicky peripherals and rehashed sequels till the cows come home.

    • Onetazien

      Who said anything about innovative? It’s just an enhancement. You sound like a jealous hater.

      • snake0

        RTFA faggot

  • mptp

    Hehe that video brings back memories – I remember hacking together a little game based on Johnny Lee’s code when I was 17. You know…five years ago.
    A little disappointing that Nintendo couldn’t see how cool this technology was back then – it took the explosion of VR over the past couple of years, a $2 Billion acquisition and their (arguably) biggest competitor Sony jumping on the bandwagon for them to hop on the VR bandwagon.

    Not to mention that as HMDs become the standard in VR, solutions like this will seem gimmicky and uninteresting by comparison. Looking through a window into another world is just not on the same level as actually feeling like you’re inside that world.

    Nintendo saying that VR is too solitary seems to me a very narrow-minded and short-sighted way of looking at things. If VR is too solitary for you Nintendo, then make it social. If anyone can lead the way into a new avenue of open, multiplayer VR experiences, it would be Nintendo.

    …or Facebook.

    • mptp

      bandwagon bandwagon

  • Nick vB

    I wonder what it is they think they can patent exactly?

    People have been using “tracking glasses” like this for years,

    That FaceOff tracking mod for Half Life 2 comes to mind, also…

    http://www.immersix.com/

  • spark

    Six degree of freedom all optical tracking with dual cameras is not a new idea.

    It’s incredible that Nintendo could seek and obtain a patent for this so-called invention.

    There is extensive prior art which substantially invalidates Nintendos ambit claims.

    What is even more amazing is how their extensive prior art and literature search failed to find such evidence.

    A good place to start would be googling ” 6dof all optical head tracking wii remote”.

    Nintendo is a company that prides itself on innovation, and of suing for patent infringement.

    This is embarassing.

  • Darshan Gayake

    Nintendo try to bring Perspective (That too, old tech nothing their own innovation) not depth so how could this mean bringing 3D at first place.

    This tech just gives sense of perspective and no sense of ‘depth'(requirement to be qualified as 3D) ……hence title for article is invalid…..!!!!!

    In in innovation here, Shame on you Nintendo.

    • QuantumElectronicsDude

      Actually, the shifting perspective as you move your head is a much more comfortable illusion of 3d than any other 3d I’ve ever seen. “Pop out” 3d bothers a lot of people. But I’ve seen this perspective-based “pop in” 3d and it’s both comfortable and looks and feels like 3d.