DisplayLink has been building out its wireless VR compression technology for the last few years, eventually finding its way into the recently released Vive Wireless Adapter. But VR headsets are inevitably moving toward higher resolutions—as we’ve seen this week at CES 2019—making it increasingly harder to make them wireless. Luckily DisplayLink has a few tricks up its sleeve to boost compression efficiency without impacting latency, one of which they demonstrated exclusively to Road to VR this week at CES 2019.

Many readers of Road to VR will be familiar with the concept of foveated rendering: since our eyes only see in high fidelity within a few degrees of the center of our field of view, it’s possible to achieve higher quality (or more efficiency) by rendering lower quality imagery in the peripheral regions and higher quality in the very center. Eye-tracking then can be used to make sure that the high quality region always stays at the center no matter how you move your eye. Done right (with good eye-tracking and smart rendering algorithms) this can be completely invisible to the end users.

DisplayLink is using this same concept, except for compression instead of rendering. Compression is critical to wireless VR because you need to be able to send high resolution imagery at high framerates over a wireless connection that sometimes experiences sudden drops in bandwidth due to the imperfect nature of wireless connections.

Maintaining smooth visuals is key to preventing freezing and stuttering which would hamper any wireless VR experience. To maintain consistency in the imagery, DisplayLink has designed their compression technology to be able to respond to changes in bandwidth on the fly—even in the middle of a frame—so that if, for instance, a user’s hand briefly blocks an antenna, the system can apply more compression to make sure the image can fit over the reduced bandwidth.

Beyond just maintaining consistent imagery under situations of reduced bandwidth, there’s also the need to be more efficient with compression so that larger frames for future headsets with higher resolution can fit in the same available bandwidth.

Both reasons are why DisplayLink is developing foveated compression which takes advantage of eye-tracking data to understand where to compress the frame the most and where to leave it sharp. In doing so, the company claims some pretty huge gains in compression efficiency.

Photo by Road to VR

Using a Vive headset equipped with Tobii eye-tracking, DisplayLink showed me a demo using their wireless adapter reference design. Initially the adapter was set to use the same amount of bandwidth available in the Vive Wireless Adapter to get the image to the headset. Then they turned on foveated compression and cut the available bandwidth down to 1/3.

To my eyes the difference between the full bandwidth image and the 1/3 bandwidth image (with foveated compression) looked effectively identical. Even as I raced my eyes around the scene in an effort to catch the edges of the more highly compressed regions, I was wasn’t able to see anything more than a fleeting glimpse of a slightly blocky region in my peripheral, and this is as I was actively trying to spot any visual artifacts.

SEE ALSO
Eye-tracking is a Game Changer for VR That Goes Far Beyond Foveated Rendering

Even when I asked them to switch rapidly back and forth between full bandwidth mode and 1/3 bandwidth mode with foveated compression, it was difficult to spot any meaningful differences between the two. Had they handed me the headset from the start with foveated compression enabled (without telling me), I don’t think I’d have any idea it was happening, and that’s exactly how it should work.

Photo by Road to VR

There’s a few caveats of course: this was a proof of concept demo and I only got to see one piece of content (just standing around in the SteamVR home area). So I don’t know if this foveated compression approach will be valid for all or even most content. It’s possible that it won’t work so well with more complex colors, contrast, and lots of motion. The demo I saw was also on the original Vive, which has a fairly low resolution compared to what else is out there. Higher resolution (like the Vive Pro Eye, which DisplayLink plans to support, and even better lenses) might make it harder to hide the foveated compression.

However, the foveated compression is fundamentally based on DisplayLink’s existing compression technology, which works pretty darn well for today’s VR headsets, so I wouldn’t be surprised to find the the foveated approach works well too.

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • I really hope this helps drag Foveated Rendering out of the concept stage and into reality. There’s no need to render the full, high-rez image at all before sending. Saves on compression and saves on GPU time.

    • Andrew Jakobs

      Except this isn’t foveated rendering.. this happens AFTER the rendering. So I wonder how it will hold up after actual foveated rendering.

      • You didn’t read my post very well.

        Like I said, it would save on all accounts if actual Foveated Rendering was used BEFORE the transmission compression, GPU and wireless transmission. Compressing an image that has already been downscaled would get an even better compression rate. Large, single-color blocks are easy to compress.

        Foveated Rendering was a hot topic in 2016 and before, but has languished since then. Just the existence of eye tracking, in even concept models, is rare now. HMD makers haven’t pushed eye tracking, instead focusing on low-end headsets, and GPU makers aren’t in a rush to setup Foveated Rendering when no commercial hardware exists to make use of it. The FOVE never made it to market and the VIVE Eye appears like it will be the first.

    • Florence S. Nielson

      watch Vice HD 2018 (*available 1080p*).with the best quality …………….
      click here –> xmovieshd-1.blogspot.com

  • MosBen

    That’s pretty impressive sounding. As someone that’s more skeptical than most about exactly how much greater the resolution of our HMDs need to get before other concerns become more important (I’m not saying that I don’t want higher resolution, just that I’d take, say, wireless 4k over tethered 8k), this sounds like it will be a practical solution for at least a generation or two.

    • Mradr

      No you wouldn’t. Wireless alone adds 250 – 300$ on TOP of the headset. Most people would rather stay with the cord or throw that money back into the screens or other part of the hardware instead. This is 100% a optional addon then it would be for something that was force onto the user.

      • MosBen

        Well first, you can’t really tell me what I’d prefer. I know that adding wireless adds cost to an HMD (though I think that you overstate the cost, which would be different if it were built in rather than an aftermarket product), and the freedom that wireless gives is far preferable to me than a bump in resolution.

        As for most people, if you really mean most general consumers, then most people don’t want a tethered (wirelessly or physically) HMD in the first place. Most people don’t own gaming computers and aren’t interested in buying or building one. But even if we limit “most people” to “most people who own gaming PCs”, I doubt that you are correct. Higher resolution is fine, but as it improves other factors become at least as important, or more important.

        • Mradr

          Yes I can. I just did. Wasn’t hard to do so. First off – Not really – wireless at that level cost more than your basic router as it has to hit speeds less than 1ms plus have a lot of bandwidth to spare. You need also 2 devices to trans/rev the data adding cost. Then you also have to warrant the device in both UK (2 years) and US (1 year). Plus if it is built in – that is just another part they could fail and then you have to send in your whole headset. You also have battery limits that won’t be replaceable, extra weight to the HMD, and a bunch of other issues. Ideally they might go from 200-250 down to 150-200. Still a pretty high extra price tag that doesn’t really add much value to the overall headset. Tether usually doesn’t bother anyone. It’s just having the PC on the other hand that bothers anyone either way. Its more of a nice thing to have in the future feature than it is something that is needed right now.

          That same cost could go into eye tracking instead and as I said above offer a lot more usefulness than any wireless will add for the given value of return. For example, allowing lower end systems (as you pointed out) to be attached to the device and offer reasonable performance to that of a console. Not wireless – but teterless. Wireless isn’t what most people want – its either an all in one to keep cost low to owning VR, or on the other side of the stream, higher visual eye candy for that people that don’t mind spending the extra money for it.

          For the few that wants wireless – there are options for them now and they should look into that instead of they have the extra cash to do so. I mean that sounds fair in all reality in terms of what the market wants and what people may prefer over all.

          • MosBen

            Well, I should have said, “You can’t really tell me what I’d prefer without being an obnoxious twat and a likely being wrong.” But then, you knew that. You’re right that adding wireless, and a battery, would add cost to an HMD. I’m not at all confident in your numbers, as I don’t think that we can extrapolate much from the prices of standalone unites like the TP-Link.

            I just think that you’re wrong on wireless not adding value or people not being bothered by the tether. People that I demo for nearly get tripped up all the time, especially when they’re starting out, and even after they get used to it it’s something that you have to be cognizant of, and it would be great if you didn’t have to worry about it. That said, you’re right that it’s owning a gaming PC that’s the biggest hurdle, and maybe it’s the case that group of people that are willing to own and maintain a gaming PC are also just more likely than the general public to put up with a physical tether. That could certainly be true.

            You’re also right that one of the nicer aspects of eye tracking is lowering the system specs necessary for a quality VR experience. I am going to quibble with you insofar as an HMD connected to an external computing platform (be it a PC, laptop, iPad, or breakout box) is what we should call “tethered” in that it needs to be connected to something external in order to function. A Vive with the TP-Link adapter is wireless, but tethered, while the Quest is wireless and untethered because all of the hardware is inside it.

            I still think that you overestimate the cost of adding wireless and undervalue it as a selling feature, but I suppose we’ll see how things shake out with the next generation of HMDs coming in the next few years. MS in particular seemed to want to hold off on doing VR on the Xbox unless it was wireless, but maybe they’ll change their mind (or their statements were misinterpreted).

          • Mradr

            Maybe but looking back over the wireless tech it’s already causing problems trying to keep up with current generation hardware let alone looking to the future of 8k by 8k screens.

            The other problem with wireless is that it took them what we already have had for a while now for 4k TV streaming just to do what 1st generation 1080p screens can do. Eye tracking FOV over wireless will help a ton I am sure. Maybe as much as 50% – allowing them another generation or two to maybe keep up with. The problem is what happens after that? Having it built in means you will have to keep up with each generation as removing a feature is usually consider a nono in most customer heads. This is just another reason to just let 3rd party companies take on lead to get this working than try to force it into the headset alone.

            Rather – as I said – eye tracking, focal point fova, and a bunch of other features seem like it’ll give you more a bank for a buck compare to wanting or preferring wireless inside the headset and overall – there are many other things HMD needs to focus on their time on let alone customer money to upgrade the HMD that covers more usage.

            At some point – mobile hardware + connecting to a PC will be a thing – so you will have tetherless feature either way for when you don’t want to be connected to a PC while still giving you the option to connect to a more powerful device later on. That will cover many fields of usage. Honestly – I am surprised Vive, Steam, or Oculus hasn’t done that already as it would open the doors for a lot more customers while keeping the number of products you make simple. Just sell the mobile hardware as an addon or upgradeable option.

          • MosBen

            But isn’t that the point of this article? Displaylink has apparently made some advances that allow for much better wireless, conceivably allowing the tech to keep up for at least another generation of display upgrades, which is why I originally said that I’d take wireless at 4k over wired at 8k.

            As for mobile devices that connect to a PC, we’re seeing the early version of that with Qualcomm’s newest reference HMD design, which has an SoC built in but can evidently connect to a PC with a detachable wire. I’m sure that Oculus and HTC haven’t done that yet because it adds cost to the HMD and it’s not clear, to me at least, that it’s a feature that would attract a lot of new customers, at least not yet. Or maybe Qualcomm just figured out how to do it and the Quest and Cosmos were already feature locked and will get it in the next generation.

            As I said in my last comment, it’s possible that the audience that’s wanting to do PC VR is simply willing to put up with wires in a way that a general audience isn’t, so wired VR on PC will just continue, while mobile VR like the Quest will attract a broader audience of people who are less interested in raw horsepower. I still think that many people in these parts underestimate the value of being wireless, but they could be right about it specifically in the PC VR space.

      • PJ

        Not me, I would a take wireless 4K headset, even if it had to restricted at 60hz over anything with a cord

  • TareX

    If this technology only handles compression and not rendering, then that’s excellent news because we don’t have to wait for NVIDIA and each developer to updated their game, and we can just use this for ALL titles (since video, is video and has nothing to do with the title) till foveated rendering is a thing.

    • benz145

      Yup that’s how it works. The frame is compressed after it’s rendered so I don’t believe applications need to do anything special.

      • Hivemind9000

        The network frame compression only really helps the wireless aspects – which means it makes no difference except to help the wireless unit perform better (higher network fps and/or supporting bigger resolutions without dropping network frames).
        Unless foveated rendering is implemented at the graphics renderer level, there will be no benefits to the rest of the VR hardware/software stack. Game developers shouldn’t have to do anything if foveated rendering implemented at the graphics driver/API level (i.e. inside SteamVR for example).
        Don’t get me wrong, I think this is a great technology, but we really need the graphics foveated rendering to support higher resolution and FOV headsets on existing GPUs. Headsets having eye tracking built in is the key technological fulcrum here.

  • Amazing. I had never thought of this application of eye tracking… love it

  • kontis

    Optic nerve has something like 1/10000th the bandwidth of HDMI, so we have a looong road of possible compression optimizations.

    • JustNiz

      That’s not true at all. Apart from anything else, the optic nerve is analog not digital.

      • G-man

        so?

  • psuedonymous

    It’s somewhat telling that the comparison was to a compressed stream rather than to an uncompressed (i.e. direct wired) image. “We didn’t make compression /worse/ is nice, but that’s not the standard to meet.

    • Caven

      At least to my eye, whatever compression DisplayLink is currently using is already indistinguishable from a wired connection. So if their foveated compression looks as good as their standard compression, it will also look as good as a wired connection.

  • oompah

    FOVeation is the correct way to do things
    instead of stupidly painting all pixels the same way
    which entails over heating, low fps , latency etc all problems
    we should learn from nature

    • Fabian

      I’m pretty sure nature is still there when you don’t look at it :)

    • JustNiz

      > FOVeation is the correct way to do things

      No it isn’t. The real world doesn’t change its resolution depending where you look.

      Foveated rendering will be an unavoidably noticable “effect” and will always look/feel wierd compared to fully rendering everything, partly because eye-tracking will always have some latency and can never be 100% accurate.