Tis the season of 5G, the next-gen mobile data technology which promises to boost bandwidth and reduce latency. With 5G phones and networks slowly starting to become a reality, carriers would have you believe that the tech is going to radically change the world, and then some. But what does 5G actually mean for VR & AR?

As the cellular industry’s hottest new technology (and hopeful revenue driver), every stakeholder from cell phone makers to chip manufacturers is playing up 5G as the start of a revolutionary shift—and what better way to do so than to associate it with buzz-worthy technologies?

You can Google “5G” along with any other hot technology in the same query and you can find someone telling you that somehow and at some point 5G is going to revolutionize that thing. I’m not even joking. Let’s take a quick survey:

The list goes on and on, and it of course includes 5G + VR and 5G + AR, with terrible articles like this one that fundamentally butcher the relationship between the technologies with sentences like these:

AR and VR depend on decreased latency to run efficiently. The user experience will be amazing with a reduced lag time. With a better UX comes more opportunity. Telemedicine, virtual training environments for business and more will be the result.

In an effort to combat this hand-waving nonsense, let’s talk about what 5G could actually mean for VR and AR. The easy way to do that is to first understand what 5G actually brings to the table.

What Does 5G Mean For You

Quite simply: 5G promises greater bandwidth and lower latency than current mobile data connections offer. It also has the potential to make such high performance networks more accessible to more people.

Exactly how fast and what latency 5G will provide to individual end users won’t be clear until the technology is actually rolled out at scale, but mobile industry marketing would have you believe that 1Gbps download bandwidth and 1ms latency is possible.

To quickly put that into perspective, the average mobile and home connection speed in the US was 33Mbps and 112Mbps, respectively, in January 2019, according to internet metrics company Ookla. As for latency, 35ms for mobile and 5-15ms for home connections are common. Roughly speaking then, 5G could offer 30x and 9x the bandwidth of mobile and home connections, respectively, with 35x and 10x faster latency.

But what in the realm of VR and AR is actually enabled or enhanced with more bandwidth and less latency?

Immersive Video Streaming

Image courtesy NextVR

When it comes to streaming non-interactive 360 or 180 video content to VR and AR devices, one major misconception is that 5G’s low latency will be a huge benefit. In fact, the opposite is much more likely to be true—it’s 5G’s big bandwidth which opens the door to significantly improved quality which would not only improve the immersive video experience of today, but also offer a pathway for further growth as VR and AR displays become higher resolution.

The reason latency hardly matters for non-interactive streaming video content is that headtracking on VR and AR headsets is not coupled to the streaming latency. In most incarnations of 180 and 360 immersive streaming technology, the visual adjustments from the movement of the user’s head are processed locally on the device at a high rate, rather than sending the motion to some remote server and waiting to receiving an updated frame which corrects for head motion.

In fact, on today’s VR headsets you could be in the middle of watching a 360 video and then completely lose connection and the only impact would be that the video would freeze—moving your head would still respond just fine because the scene’s movement with regard to your head is processed right on the device.

There’s some nuance to this: some more sophisticated VR and AR streaming uses ‘view-dependent’ technology which optimizes the quality of the footage specifically in the area that you’re looking (in an effort to minimize the bandwidth bottleneck). High latency in this case could mean that if you turn your head quickly to a new part of the scene, you might notice that the video isn’t as sharp until the higher quality section of the video pops into view. In this case lower latency will make those transitions smoother, but it’s more of bonus than a radical shift in the experience.

When it comes to bandwidth however, 5G could significantly improve 180 and 360 video streaming quality. The most immersive footage—360 3D or full volumetric video—is also the most bandwidth intensive and is used sparingly today in a large part because most users don’t have sufficient connections to stream this kind of content in high quality.

Cloud-rendered VR and AR Gaming

Image courtesy Microsoft

When it comes to interactive VR content, 5G could make cloud-rendered VR and AR gaming possible, but this use-case is still largely hypothetical.

The pitch goes: users will own a simple inexpensive headset that streams high-end PC VR or AR graphics right from the cloud with no PC needed! It sounds amazing, but you aren’t going to see this happen until well after normal non-VR game streaming matures because VR demands significantly higher resolutions and lower latency than non-VR game streaming (which is still struggling to meet the performance needs of competitive gamers).

While latency isn’t really that important in the case of non-interactive video streaming, here it is crucial because what’s being rendered needs to change in real-time based on the actions of the user. So if 5G can truly offer up, let’s say realistically, 4ms or less latency, it really could unlock the cloud-rendered use-case of VR and AR.

But that’s a big if. Not because sub-4ms latency isn’t possible, but because you need that level of latency not just to your nearest connection in the network, but specifically to the location where your frames are being rendered.

If you run a network speed test right now, the test will very likely connect to a server that’s very close to you. Your latency is the round-trip time that it takes to get information to that server and back. If you test again and select a server that’s a few states over, you’ll find something very different. I just did this exercise and saw 10ms of latency to a server in my city and 35ms to a server that’s three states over.

This is to demonstrate that network latency increases as a function of your physical distance to the servers you are pinging. If you want to render VR and AR content in the cloud at low latency, the user needs to be physically close to the location where the frames are being rendered. This is much more difficult to achieve at scale because the frames of an interactive application have to be rendered on the fly.

When it comes to non-interactive video streaming, it’s relatively trivial to distribute the pre-rendered video files to servers across the globe ahead of time, and then serve them over the network as needed. Real-time rendering of VR content requires high-end GPU hardware in each data center from which cloud rendering would happen; each properly equipped data center can only deliver those frames with low enough latency to users who are within a certain physical distance.

That is to say that a single data center in the middle of the continental US, for instance, would have too much latency by the time it reaches the country’s coasts for a viable cloud-rendered experience experience. A distributed cluster of capable data centers (also known as ‘edge computing’) is the key to making this viable; 5G helps by expanding the ‘coverage’ of capable data centers by reducing latency, while high bandwidth makes greater quality imagery possible in this use-case (which will be increasingly important as the resolution of VR and AR devices increase).

Though it may take both 5G and edge computing to make this all possible, the good news is that cloud-rendered VR and AR services can piggyback off the same infrastructure that’s being deployed for non-VR game streaming services like Google’s Stadia and Microsoft’s xCloud.

Continued on Page 2: Telepresence and Remote Control »

1
2

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • JesuSaveSouls

    Jesusavesouls !

    • Trenix

      Can you stop being a mockery of the Christian religion? You’re not bringing awareness, you’re making people convert into Atheism.

    • Graham J ⭐️

      Jesus gobbles sausage

  • Graham J ⭐️

    Great article; nice to see some reality checking on 5G. I’m surprised you didn’t mention AR more though. While there’s probably even less reason to remotely render frames for AR than VR, it’ll likely be accessing large datasets such as point clouds. Also way more likely to be mobile.

    • dk

      fast internet connection for ar will be a must have if a lot of people want to see the same thing …especially when interacting with that thing ….like ar laser tag arena with 100 people with virtual obstacles and avatars and some skeleton npcs or whatever
      ….the old simple way with a few hololenses was one hololens is like the server coordinating everything and a few can join in with interactive app

      • Graham J ⭐️

        That’s a great point. If multiple people see an overlay at a static position then the latency of the apperence of the overlay would be the server to observer latency. But worse, if the overlay is over a remotely tracked object (like another user) there would be additional positional latency.

    • Andres Velasco

      Here a building burnt killing 75 people triggered by 5G

      https://www.youtube.com/watch?v=igFNYM8RWZU

      • Graham J ⭐️

        That was caused by a kitchen fire and spread because of lax laws around building cladding. And it was in 2017. You’re a gullible fool.

        • Andres Velasco

          Yes it was aluminum. High frequencies and metals don’t do too well together on some scenarios or so that are the discussion. Lol. You are an absolutist. Dont be so obtuse

          • Graham J ⭐️

            Is that what the nutters are saying, that signals from a transmitter on top of the building interacted with aluminum in someone’s kitchen, starting a fire? And you believe that? lol

  • Andres Velasco

    I will stay away from 5G, but will pay attention how it’s used

    • bud01

      We are not going to be able to,

      Its been worked on for more than a decade yet there is not one serious study showing it is safe and in fact all public studys before the FCC clamped down on it being open to question showed it caused major and minor health concerns, it effects the blood brain barrier, it is bio active, ionizing, huge impact on the Circadian rhythm.

      See : https://www.theepochtimes.com/5g-is-the-stupidest-idea-in-the-history-of-the-world-says-professor-emeritus-from-wsu_2882838.html

      We are made of cells each which vibrate, any and all EMF cause stress and deplete reserves of vitality (if not worse, dna damage, mutations etc).

      Man needs both the benefits of 4G/5g but less of the downside related to mass EMF around what is organic material.

      Common sense comes into it at some point.

      Have consideration for both the technology, what it is doing and how it is doing it, and also have compassion consideration of what we are made of.

      5G Is The ‘Stupidest Idea’ in the History of the World, Says Professor Emeritus From WSU, there is a global army of scientists going crazy about this topic.

      If you listen to some one discuss this topic that knows it well, it is an open and shut case this could or should in no way be deployed.

      There needs to be a consideration of mass clothing availability for EMF shielding this was discussed here a littlle:
      https://www.coasttocoastam.com/show/2019/09/19

      Its getting rolled out regardless, even if gxd said to stop.

      4g doubled the myopia rate in children lets see if 5g can quadruple it

      Google keyword: Myopia increase in children

      You put a RX TX high powered Microwave transmitter by your eyes balls, sure there is no impact… this is what an entire generation behave as if, including getting into “trains” / “faraday cages” and stream content on mass, 10,20,30 units a train or more with all the considerations around harmonics, hot spots, totally no place…

      Its amusing as a previous radio engineer to see,
      At the very minimum you are mass dull the well being and vitality of a generation.

      Humanity and its technology’s are still evolving, this is very raw crude, like going back to Victorian hospital and watching the doctors working.

      • Jistuce

        You’re quoting some wild anti-science nonsense. Hell, your linked article calls cell phones “radioactive” in the opening sentence. It also swaps around between “proven harmful” and “never tested on humans” depending entirely on which stance is more supportive of the argument it is making. And uses the phrase “EMR poisoning”, which is a bad joke(poisons are chemical compounds. It is why people exposed to high levels of particle radiation have “radiation sickness” and not “radiation poisoning”).

        “4G” had no effect on myopia, and if you want to claim otherwise, you need to find a plausible mechanism by which 700-2100 MHz radio waves can cause nearsightedness as well as an explanation for why it only happens when using 4G data encoding(as the frequencies have been in widespread use for a long time).

        Also, cell phones are not high-power microwave transmitters. They are, in fact, quite low-power radio transmitters.

        • bud01

          See:
          CDPH – How to Reduce Exposure to Radiofrequency Energy from Cell Phones

          Google keywords:
          Prominent scientists warn that 5G could pose health hazards

          Enjoy the rest of your day.

          • aasdfa

            you can google ANY keywords and find crap articles supporting whatever you typed in. shit some are auto-generated….

            this is why anti-vaxing is a thing. Just because someone put it on a website doesnt make it true.

            Enjoy the rest of your life being insanely gullible to the point of destruction.

          • bud01

            Keyword
            The inconvenient truth about cancer and mobile phones.
            (Guardian)
            See
            Petition :26,000 scientists oppose 5g roll out.

            Just because some thing is available does not make it safe,

            Cars contaminate air which you walk by, this was and is insanity.

            Aspartame is in the food chain, a known neurotoxinL.
            tobacco has several unhealthy additives.

            When the guy pushing for Galaxion looks at something sideways that is tech related, best to ask questions as an open cup.

            Measure twice cut once, don’t cut once then measure.

            Health implications or concerns no place in the progression of this is a finger print clue of pro / negative .

            Do your own research.

          • care package

            Just because someone put it on a website doesn’t make it untrue either.

          • Jistuce

            Which is why one must read with a critical eye.

            (The anti-vaccination crap is exceptionally galling, since it is literally killing people.)

          • care package

            Everyone thinks they read with a critical eye, yet we all disagree. Why? Because no one gets past their bias, or what they want to believe. You either have a love for truth or you don’t. Most don’t.

        • care package

          Just as there is always someone non trusting, on the other side of the coin you’ve the complacent humanists that refuse to think anything other than the Star Trek utopia is slowly on it’s way.

          • Jistuce

            There’s a difference between thinking that everything and everyone is inherently good, and thinking that crank pseudoscience should be pointed out for the snake oil that it is.

            I can very confidently say that 4G did not cause myopia, there is no such thing as EMR poisoning, and cellphones are not high-power transmitters of ANYTHING without also being naive and trusting of everything with the word science attached to it.

          • care package

            Sounds like you are just talking out your ass really. This isn’t about 4G.

          • Jistuce

            In part it is. I was replying to a post ranting, among other things, how 4G made kids go blind(but 3G didn’t, despite existing at the same power level and frequency).
            And half of 5G(the half you’re going to see in the wild) is on the same frequencies as 4G, so anything applying to 4G also applies to that.

      • MasterElwood

        Troll or idiot? Hmmm… Well – doesn’t matter. Blocked.

        • bud01

          From the generation that will have the major buying power in the worlds markets in five years reading this is scary.

      • Andres Velasco

        Yes, I agree with you. Most people that down vote the dangers are people that are mainly on their selfish desires for gaining some form of benefit, while disregarding any dangers

        • bud01

          Actually that’s not the case, its the opposite. The leaders in insurance and leaders in telecoms at the European and American fund level are asking a lot of questions, there are a lot of quality people out there, you would be surprised, they are informed and when they are not they know it and are proactive in reaching out for correct information from ALL source not just BB FT, Reuters .

          Insurance people in particular are a little worried to the point of just laughing about it as unbelievable with a fearful look on their face, dear in the headlights. BIAS or preference means little when professionals look at some thing and high quality choice has to be made.

          The ideal would be roll out 5g, stuff like IOTA, whole internet up to ipng (v6) man kinds progress at accelerated pace since it was stunted, but if people start getting massively impacted, not just blurry eyes, dulled sense of well being, mental focus, slight impact on cellular vitality, but actual full bore major health impacts which are directly measurable. That’s not going to work, the internet enables mass information sharing and communication, 5G already has a red flag over it, the data is going to be being watched closely, you tube will exploded with a million videos, face book,

          People like to go strong both directions of like recommend and other side of the stick.

          With are beings which are electrical first then chemical.

          If 5G has to roll out it has to roll out end of story, but people will need options to alleviate or remove any major or minor irritation.

        • Graham J ⭐️

          And then there are people who actually understand the science and form their worldview based on evidence instead of crackpot theories.

          You don’t know “most people”.

      • care package

        Were all f’d with out without 5G.

        • Graham J ⭐️

          Welp, might as well kill yourself then.

          • care package

            Great Idea Graham, but not my life to take.

          • Graham J ⭐️

            Oh I see, you’re a god nutter. Figures.

          • care package

            Oh I see, you’re a god hater. Explains why you love death, and a wanna-be know it all. Figures.

          • Graham J ⭐️

            I cannot hate something that does not exist.

          • care package

            Your belief has no effect on reality nor will it save you from eternal hell. Life coming from nothing/no life is whats never existed. Everyone knows life is a creation. Some just choose to be in denial. If one could not hate what they don’t believe in, then anti-theists pretending to be atheists wouldn’t exist.

          • Graham J ⭐️

            No one’s beliefs have any effect on reality; why would they? Life formed because conditions were amenable to it but that is irrelevant because the universe existed for billions of years before that. Some choose to make up reasons for the universe being the way it is but rational people follow the evidence.

          • care package

            Your rational, just ask you lol. Nice job preaching your belief system/religion there though. There is no evidence the earth is billions of years old, nor is there regarding your fantasy ‘life formed because conditions were amenable to it’. That takes more faith than believing life was a creation.

          • Graham J ⭐️

            Of course there is evidence. Learn science. You’re the one preaching a belief system.

          • care package

            Love it when evo’s just play the ‘science’ card. Evolution has nothing to do with science. Science is observational, and as far as the ‘science’ behind ‘billions of years’ radio dating methods are all extremely flawed. Results are always all over the place and the preferred one is cherry picked. Science is also repeatable, which radio dating is too inconsistent to be reliable. Its still all pushed because most in this world hate truth.
            As far as preaching hell yes I am, and so does every evo proclaiming their faith.

          • Graham J ⭐️

            Who said anything about radioisotope dating? Although I love the irony of someone who believes the universe was created by some magical sky being complaining about the accuracy of measurement tools, all one needs is a telescope. Or do you not believe in refractive optics or the speed of light either?

            Science isn’t a card, it’s the study of reality. I guess it’s no surprise you don’t understand it.

          • care package

            I said something about radioisotope dating, or did you not catch that. I brought it up because that’s what evo’s use for billions of years, and their best argument for evolution is homology. Of course you finish with an “i just don’t understand it’ jab. So far, all you’ve done is express your opinions and preached your religion offering zero actual insight. good job.

          • Graham J ⭐️

            No one uses dating to determine the age of the universe. You’re on crack. And you don’t understand science.

          • care package

            While you fail to offer any insight still, and continue to make claims. Go ahead then Graham. How do they determine it then.

          • care package

            Oh btw, I let the author know you think he’s on crack

            https://www.livescience.com/32321-how-is-earths-age-calculated.html

  • Immersive Computing

    This graph from Sridhar Sunkara at Ebiz demonstrate cloud computing very well; the take away is that laptops, smartphones, iot sensors, etc. are the Edge layer.

    One possible future is a move to “home hub” local compute (think Nas with AI) operating your IOT home infrastructure and acting as your Edge layer.

    Security, power, heating, water, internet, media server and serving high bandwidth low latency immersive computing content through 5G across your premises – removing need for PC, console, etc. in each room.

    https://uploads.disquscdn.com/images/f4f3eb0e0fa70b08fd4fc0b7590bf17291c2a4c9e4ee0e4a3b11481fbc97c2ce.png

    • Graham J ⭐️

      That’s not what edge means.

  • Miguel Arjona Villanueva

    Nice article but I have to disagree in one point regarding Cloud Rendering. Your are right when you say that latency increases with the distance, but alternatively you can use Edge Computing (or you will) to ensure that latency from the sever is less than 10ms (should be enough). We have demonstrated it in the last Vodafone 5G European Forum trasmittting a high quality of digital twin of a real factory in real time from the Edge through 5G to an Oculus Quest headset

    • benz145

      Edge computing and 5G go hand-in-hand:

      That is to say that a single data center in the middle of the continental US, for instance, would have too much latency by the time it reaches the country’s coasts for a viable cloud-rendered experience experience. A distributed cluster of capable data centers (also known as ‘edge computing’) is the key to making this viable; 5G helps by expanding the ‘coverage’ of capable data centers by reducing latency, while high bandwidth makes greater quality imagery possible in this use-case (which will be increasingly important as the resolution of VR and AR devices increase).

    • Larry

      AFAIK there are zero real world 5G tests showing sub 15 ms latency.
      All of the real world tests show 20 to 30 ms which is far too slow for
      VR. 4G latency in the US is 40 to 60 ms, it’s impressive they managed to
      cut that in half.

      Yes, there are lots of demonstrations and lab experiments with low latency 5G, but that’s not the real world.

      If you have data showing real world tests, please post it.

      • Miguel Arjona Villanueva

        Using Edge Computing the XR Rendering Server is installed a close as possible to the final user (even a few kilometers) so you can get latencies under 10 ms. Of couse, Edge Computing from Teclco is under development but the tests in real environment shows that it is (or will be) possible. In the image you can see a digital twin of a factory sent to an Oculus Quest with the same quality than HTC Vive using 5G https://uploads.disquscdn.com/images/8b01a6c6b5a6815c1bee35becc655eba4ffec4aaff83ddfa53ab201edf51ac48.png

        • Larry

          Like I said, that’s a very nice demo. Nobody is going to finance and build and operate billions of dollars worth of edge computing hardware for a very small market of a few million VR users.

          • Miguel Arjona Villanueva

            Maybe NVDIA in thinking on it: NVIDIA CloudXR (in Cloud but applicable to Edge)

  • Larry

    Real world latency testing on 5G is showing 20 to 30ms. Better than the 40 to 60ms typical in US cities but nowhere near as fast as claimed (not a surprise) and obviously far too slow round trip times for any VR application or edge rendering. The claims of sub 5ms latency on 5G are just fantasy.

    As the article says, the idea of VR + 5G is mostly overhyped marketing. VR has enough challenges without being lumped in with 5G.

    • benz145

      It depends what you’re testing. The super low latency claims are to the ‘edge’ of the network (the nearest node). That’s why edge computing is such a key part of making this possible.

      • Larry

        Yes, that’s the vision Intel pushed when they were in the 5G space. What they don’t say is that deploying that level of computing to the ‘edge’ will cost many billions and require data centers everywhere. It’s also completely backwards to the current model of data centers which is to put them centrally where power and real estate is cheap. VR itself is niche enough with only a few million users, no company is willing to invest billions in edge rendering on the very small chance that this will suddenly appeal to consumers. If VR was so compelling, more people would be buying the Quest etc.

        • benz145

          Yup, that’s a central point here; edge computing is necessary and the economics need to make sense too. Granted, it looks like there’s a lot of money being poured into non-VR game streaming services, and these will need to make use of edge computing too if they’re going to take off with hardcore gamers; VR and AR cloud-rendering will likely benefit from those investments.

      • Baldrickk

        When Steam link over local wired gigabit at only 1080p isn’t fast enough to not overcorrect in flat screen racing games, there’s no way an internet based streaming service is ever going to be low latency enough to support VR.

        Just isn’t going to happen.

  • Rodgerroe

    Hyped for the wireless VR improvements 5G will bring. No way I’m putting up with a cable again.

  • dota

    It means this:
    u will be able to play games at very fast speed
    u will get very fast game streaming
    but
    at the same time it will cook ur brain & other organs till
    u become a zombie or cancer ridden

  • Jeremiah Tothenations

    If the UK goes with the Chinese variant of 5G then count me out.

  • Ardra Diva

    From what i’ve read WiGig is way more important in the short term. It has massive bandwidth but short range, but perfect for streaming from your PC.

  • ShaneMcGrath

    Look, No matter how much they want to sell and push this “cloud” anything, I don’t want it!
    Rather have it at home to use when I want.
    Just had ISP issues last week(yet again) no internet for 4 days,May as well have been the stone age,No online banking,No news,no nothing.

    90% of my games were unplayable, Luckily I still had some offline and GOG games to keep me going.
    Screw cloud gaming and subscription based gaming!