7invensun, one of the latest companies accepted into HTC’s Vive X accelerator program, is launching an eye-tracking add-on made specially for the Vive headset. The device, which will go on sale initially as a development kit in China this month, uses a series of IR LEDs and a near-eye camera to track the player’s eyes in real-time, enabling a range of potential benefits for VR.

Meeting with 7invensun at HTC’s Vive X accelerator space in San Francisco today, I got to see the aGlass eye-tracking development kit in action. The device, which comes as two compact assemblies which easily insert around the Vive’s lenses, uses a ring of IR LED’s to illuminate each eye while a small camera records the movements and sends the data back to the computer for processing. The devices are smartly powered from the Vive’s extra USB port that’s hidden under the cable retainer at the top of the headset.

Foveated Rendering and More

Eye-tracking can be used for a number of useful purposes in VR, perhaps one of the biggest being foveated rendering, which reduces the resolution around your peripheral vision where it doesn’t need to be as sharp, while leaving the center part of your vision at full quality. This can significantly reduce the processing power required to render a high-quality, high-framerate VR scene. Among the VR industry, foveated rendering via eye-tracking is largely considered a necessary technology to one day achieve retinal-resolutions in VR headsets by reducing the processing power needed to render on ultra-high pixel density displays.

SEE ALSO
Vive Deluxe Audio Strap Gets June 6th Release Date

Foveated rendering is already up and running with the aGlass dev kit, and is presently said to work on any application with no modifications, so long as the computer is equipped with an NVIDIA GPU. The company will aim to extend the capability to AMD cards as well. My current understanding is that other functions that employ aGlass’s eye-tracking data are GPU-independent, but foveated rendering is a special case because of the current rendering technique.

Beyond foveated rendering, eye-tracking can be used to more realistically animate the eyes of your VR avatar, simulate depth-of-field effects, be used for conscious and unconscious input inside of VR apps, and more.

Corrective Lenses

Beyond eye-tracking, the aGlass add-on also opens the door to corrective lenses for those who normally wear glasses. Today’s high-end VR headsets lack variable focus control, so those with glasses are either stuck trying to fit them inside the headset or not playing with them at all (and suffering a blurry virtual image). The aGlass add-on is designed for easy insertion of custom-made lenses that rest over the Vive’s original lenses, and the development kit comes with three focus powers. And if you don’t need corrective lenses, the device works entirely fine without them.

Hands-on With the aGlass Dev Kit

I got to try the aGlass dev kit myself at the Vive X accelerator office in San Francisco today.

The calibration process took several tries and quite a bit of fidgeting with the headset to get right. In my experience this is not entirely unique to aGlass, but speaks to the underlying challenges of designing a robust eye-tracking system that can handle a wide range of face and eye structures, not to mention those with eye problems (just an FYI, I’m fortunate to have normal 20/20 vision).

SEE ALSO
Eye-tracking Glasses Give Glimpse into Professional Pianist's Perception

Once we did get the system calibrated—which involves looking at a series of dots in the screen and takes only a few seconds, if it works in the first try—foveated rendering was demonstrated using NVIDIA’s VR Funhouse. Foveated rendering only works well if the eye-tracking is fast and accurate enough that it’s hard for the user to notice any foveated rendering happening.

I was told the aGlass has a 5ms latency, and this was fast enough that—under the right rendering settings—I could barely tell that the foveated rendering was turned on, which is great. The particular method they were using to achieve foveated rendering was described as “proprietary,” and it did work better than a number of other attempts I’ve seen, which often introduce a blurry feeling in the periphery which gives the effect away easily and doesn’t feel quite right.

The particular laptop computer powering this demo lacked enough power to be formally called ‘VR ready’, and when foveated rendering was disabled I could easily see lots of jitter due to the computer not being able to keep up with the rendering workload. Once foveated rendering was enabled, it allowed the computer to achieve the necessary 90FPS consistently.

Improvements to Come

The installation seems dead simple, though the current dev kit adds a little bulk around the Vive’s lenses which slightly restricts the IPD setting on the lower end. The average IPD is often quoted around ~64mm, but with the aGlass installed I was only able to bring the Vive IPD down to 65mm, though the company says they aim to fix this for the consumer version of the device.

SEE ALSO
Oculus Research Reveals New Multi-focal Display Tech

Another issue for me was the need to dial the Vive’s lens-relief out a few notches to get the calibration to work correctly. Doing so means reducing the headset’s field of view which is of course no ideal. The problem may be due to the angle of the eye-tracking camera which is mounted on the aGlass bezel below the eye. The company says they hope to improve the robustness of calibration by increasing the ‘sweet spot’ of the system so that it can better handle varying eye and facial structures.

On the aGlass dev kit there’s also two soft rubber flaps that I could feel gently pushing against the inside corner of my eyes. It seems these may be needed to keep the IR light from interfering with the Vive’s head-sensor (which is positioned on the headset right between your eyes). It didn’t hurt but was slightly discomforting since you don’t usually have something touching that part of your eyes. The company says that the consumer version won’t require the flaps.

aGlass Dev Kit Release Date

7invensun says the aGlass eye-tracking dev kit add-on for Vive will go on sale in China this month, priced around $220. In the US, the company expects to offer the dev kit for sale in Q3. There’s no word yet on when a consumer version of the device will become available.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • DaKangaroo

    I hope the Vive 2.0 is just basically the currently Vive with all these awesome addons built in (eye tracking, new headstrap, wireless, etc), the improvements we’ve been hearing about from Valve already (new controllers, lighter/better headset, cheaper lighthouses), plus say an increase in screen resolution (eg 1080×1200 to 1620×1800, without a pentile RGB matrix this time) while maintaining 90hz.

    All that combined would add up to a gigantic leap forward for VR in terms of quality, if Valve could put out an update like that in the first half of 2018, that’d be epic in my books.

    • Sebastien Mathieu

      yesss… completely agree… not to mention the vive tracker, future look bright indeed for pushing VR further….

    • RFC

      The recent interviews with Valve VR team members on youtube had them stating that all the new tech is “done deal” just waiting for manufacturers to bring it to market (in realistic product cycle).

      Gabe seemed especially fired up about the new display panel technology coming for VR HMD, which easily surpasses the current smartphone tech – now that panel manufacturers have a VR market with installed base to develop for.

      I had two Vive’s and thoroughly enjoyed the ‘raw’ experience, but am currently sitting it out waiting for MY18 release of 2nd generation consumer PC VR.

      Once Nvidia Volta GPU comes to market, and we get slightly higher-res HMD equipped with eye tracking for Foveated rendering, we will see big improvements in graphical fidelity which will be a huge boost for immersion

      Can see Vive 2, LG VR and other new players to SteamVR really taking flight once Valve drop their three VR games into the market

      • elev8d

        DaKangaroo & RFC, you are both completely on point. I’m doing the same. Also, waiting to see what AMD Vega brings to the table. The LG demo did a 42% increase in pixels, which wasn’t very noticeable to reviewers, so I’m hoping for 2x resolution without pentile.

        • NooYawker

          I just got a new system but I’m definitely excited to see AMD back in the game. Most gamers were big on AMD before the i7.

      • JustNiz

        >> Once Nvidia Volta GPU comes to market,
        It seems like gamer-level product won’t get full-blown Volta. It will be called Ampere and probably won’t have HBM.

    • Fam Wired

      Without pentile matrix is probably the most important feature I think. The current screen resolution is NOT 1080×1200 for Red and Blue.

      • DaKangaroo

        Absolutely. Removing the pentile matrix would not require any additional processing power at all and yet it would result in a very real increase in apparent resolution of the display. It really could of been done with the first Vive but wasn’t.

        That plus a moderate increase in resolution, say 50% extra width and height, would really make a huge difference.

        I say only a moderate increase, because there’s no point in adding an 8k display or anything like that, no point in creating an expensive display that few will have the GPU power to handle.

        • Mike

          Actually, with foveated rendering, mid-range GPUs could handle 8K. Not 16K though.

        • Mike

          Actually, with foveated rendering, mid-range GPUs could handle 8K. Not 16K though.

        • Tomas Sandven

          I think you underestimate the power of Foveated Rendering. With Foveated Rendering combined with Simultaneous Multi-Projection (SMP) I wouldn’t be surprised if a GTX 1070 could easily render to two 4K screens at 90 FPS.

        • Tomas Sandven

          I think you underestimate the power of Foveated Rendering. With Foveated Rendering combined with Simultaneous Multi-Projection (SMP) I wouldn’t be surprised if a GTX 1070 could easily render to two 4K screens at 90 FPS.

      • Adrian Meredith

        exactly, anyone whos played elite dangerous with ui modified to green can see how much higher the resolution appears. The text becomes nicely readable. If the next gen has 2k non pentile then it will be lenses that hold back the quality

    • Ethan James Trombley

      They could do two 4K screens for foveted rendering. That would still require a solid graphics card but doable. Maybe release two Vive 2s with the only diff being resolution.

    • JustNiz

      >> I hope the Vive 2.0 is just basically the currently Vive
      Urgh why? I want it to have much better lenses, much wider FOV, higher resolution, no visible SDE, built-in wireless and also much be better fitting and lighter.

  • REP

    Nah, resolution need to be AT LEAST 4k!! I don’t care how they achieve this, but it needs to be at least 4k.

    • George Vieira IV

      Foveated rendering needs to come before 4K (or at least at the same time).

    • Fam Wired

      And hopefully also RGB matrix instead of the pentile matrix to just bump the resolution of the specification papers.

      • nalex66

        As long as the screen is high-res enough that you don’t see the individual pixels, I don’t care what the sub-pixel layout is.

    • Sam Illingworth

      At what FOV? As they’ve said in articles a number of times, I think we need to stop specifying resolution in x*y and start specifying it in per degree.

    • Brent

      You can turn it up to 4k now!!

    • Buddydudeguy

      No it doesn’t. That’s where foveated rendering comes in.

      • Marc

        Well, pixel density needs to be at 1000+ PPI with Foveated rendering. Thats where the money is at… Also add 140+ degree Field of view

  • I’d be very interested in a developer kit as soon as they put in Blueprint-level support into UE4. Epic has been very good about supporting various VR features, I’m certain they’d pop that into the next release if they had the right code access.

  • Me

    This is big, I honestly didn’t believe it would be possible to add that as a kit simple as that.

  • Xron

    Waiting for Vive 2.0… Because I can’t see many advantages for now… you can buy better video card or cpu if your system is weak, instead of buying this add on.
    Though for next gen it will be super important as we get way higher res and fov (hopefully) ofc if price of hmd won’t skyrocket.

  • Angelo Overmeier

    “Doing so means reducing the headset’s field of view which is of course no ideal.”

    So yeah I wouldn’t use this at all. :D But it’s exciting that the tech seems to be almost ready and this is importan for next gen hmd’s if they want to have a 4k resolution. :)

    • elev8d

      I think it’s because the lenses and attachment go over the current lenses. If they were integrated into the hmd then they wouldn’t require as much clearance?

      • Tomas Sandven

        Indeed

      • Tomas Sandven

        Indeed

  • Taosaur

    I was surprised how well the Vive accommodates glasses. Aside from getting the headset on and off, I find very little difference in the glasses vs. contacts experience, and I have a big fat head and terrible vision.

    • elev8d

      Yeah, the eye relief settings are very important, although they do add weight to the HMD, which is one more reason why the Oculus is a little lighter.

    • Zobeid

      My glasses were starting to scratch the lenses inside the Vive, until I got a set of corrective lenses from vr-lens-lab.com. I’ve been using them for a couple of months now. My only negative was the distortion which was a bit distracting at first. . . Now they have introduced a RABS lens with less distortion, so if I were getting them today I would go for those.

  • victor

    This makes me lean towards buying Vive instead of oculus now.

    • Foreign Devil

      If you’ve waited this long to buy you are not likely to buy either of the current Generation.

      • victor

        oh yes I am and no one can stop me.
        It took me this long to work up a budget fyi

        • Flikr

          Wait like two months for us to know what NOLO VR’s like compared to the vive, and if it performs well on an OSVR HDK 2, get those instead. Saves you a couple hundred for the same games, so you have a headstart on saving for gen 2!

          • brandon9271

            That was my plan until I found a lightly used Rift + Touch on ebay for $460. Couldn’t get an HDK2 and Nolo for that. Would be cool to try the Nolo with something higher res like PiMax or GearVR and RiftCat.

        • Flikr

          Wait like two months for us to know what NOLO VR’s like compared to the vive, and if it performs well on an OSVR HDK 2, get those instead. Saves you a couple hundred for the same games, so you have a headstart on saving for gen 2!

        • KDmP_Raze

          The FOVE headset is coming out in a couple months with 1440p resolution and full eye tracking. It works with lighthouse system and vive controllers. might be worth looking at cause the current headsets REALLY need higher resolution.

        • KDmP_Raze

          The FOVE headset is coming out in a couple months with 1440p resolution and full eye tracking. It works with lighthouse system and vive controllers. might be worth looking at cause the current headsets REALLY need higher resolution.

        • NooYawker

          My advice is buy a used system if you can. The software is few and far in between and who knows when Gen 2 hardware will drop. As someone who had to have it, I know how you’re feeling. But go used if you gotta have it now. I haven’t used my Vive for awhile now.
          Just buy new face pads :)

          • victor

            Thanks but elite dangerous and warthunder and il2 is more than enough games for me (all 3 working great in VR).
            I do have life to tend to also lol

  • atg284

    “retinal-resolutions” …Please just call it what it is. High definition like 1080, 2K, or 4K. Quit using apple’s BS marketing.

    • Tim Suetens

      It’s not a marketing term. “Retinal resolution” literally refers to the resolution of the human eye, or the retina. It is a lot higher than the resolutions you listed. Higher than 8K, in fact.

      • Flikr

        What is it estimated to be? These guys are saying a mid-range gpu can handle 8k with foveated rendering, so I’d like to know if, say, a Titan XP could handle anywhere near close to human vision with it.

        • REP

          With foveate rendering, they can achieve retinal resolution no doubt. You’re talking about only a small area that need to render. VR should never have to deal with the BS SDE!! foveate rendering is the holy grail of VR!!

          • brandon9271

            It’s definitely a huge piece of the puzzle but what about a much wider FOV? High FOV + High Res + foveated rendering is what we need. Hopefully it wont take too long. The tech already exists, its just a matter of putting it all together in an affordable package. I’m afraid what we’ll ultimately get is Rift and Vive v1.1. Slight improvements and lower price but nothing Earth shattering until VR has mass adoption.

        • REP

          With foveate rendering, they can achieve retinal resolution no doubt. You’re talking about only a small area that need to render. VR should never have to deal with the BS SDE!! foveate rendering is the holy grail of VR!!

      • Jargon Houiweirz

        It is a marketing term Tim. It doesn’t exist outside f a brand name.

    • Tim Suetens

      It’s not a marketing term. “Retinal resolution” literally refers to the resolution of the human eye, or the retina. It is a lot higher than the resolutions you listed. Higher than 8K, in fact.

  • Zobeid

    quote: “Beyond eye-tracking, the aGlass add-on also opens the door to corrective lenses for those who normally wear glasses.”

    Article makes it sound like this is a new thing, but VR Lens Lab has been in business for a while now. I’ve been using their lenses in my Vive.

  • JustNiz

    It sounds like the effect would be at least subconsciously noticeable even on a gaming PC, and the trade-off of having to dial the lens relief out and so narrow your FOV alone makes it a non-option for me… Then there’s the price. $220 is a lot for this, and foveated rendering isn’t hardl;y supported yet anyway. It seems like the better option would be to save the money towards a next gen headset with this tech already built in, or a more powerful GPU that eliminates the need for foveated rendering in the first place.