At MWC Los Angeles this week, NVIDIA CEO Jensen Huang demonstrated the company’s CloudXR platform which is made to stream cloud-rendered AR and VR content over 5G connections. Built to support SteamVR/OpenVR content out of the box, Nvidia says it will release a CloudXR SDK to enable companies to offer AR and VR content from the cloud.

Nvidia wants to leverage GPU-based cloud infrastructures to enable businesses to render high-end AR and VR visuals remotely and deliver them to customers over 5G. The idea is to remove VR’s high-end hardware barrier by rendering the visuals in the cloud and streaming them to a host device which itself doesn’t need particularly beefy or expensive hardware. Nvidia already offers a very similar service called GeForce Now, but it’s for traditional games rather than VR.

Now the company says it has developed a cloud-rendering pipeline, specifically supporting SteamVR/OpenVR content, which the company is calling CloudXR. Rather than offer this service directly to customers (as with GeForce Now), Nvidia is positioning CloudXR as a set of tools which other businesses can use to bring AR/VR streaming to their customers. This approach makes sense because one of the key pieces to this puzzle is a 5G network (thanks to its potential for low latency) and Nvidia hopes that carriers who are building out 5G networks will want to offer CloudXR streaming as a way to attract customers to their networks.


Nvidia has talked about the possibility of AR/VR cloud rendering in the past, but this week the company is formally announcing an early release of the CloudXR SDK which can be used as the basis for bringing cloud-rendered AR/VR content to customers. The SDK includes:

  • Server driver that runs in the data center
  • Easy-to-use client library to enable VR/AR streaming for a multitude of OpenVR applications to Android and Windows devices
  • SDK for portable client devices that let application developers easily stream rendered content from the cloud

The system is designed to work with SteamVR/OpenVR content out of the box and be able to stream to client software running on Windows or Android, which could include Windows host PCs, Android-based standalone headsets, or even handheld devices (for handheld AR).

CloudXR Demo

On stage at MWC Los Angeles this week Nvidia demonstrated CloudXR in action in a handheld AR mode. A high fidelity 3D model of a car (rendered in the cloud) was projected onto the stage using a phone as the augmented reality platform (presumably Android, with a CloudXR client integrated with Android’s ARCore tracking).

Nvidia says the system works to “dynamically optimize streaming parameters and maximize image quality and frame rates, so XR experiences can maintain optimal quality under any network condition.” But the company isn’t talking specifics about latency requirements, other than to say that CloudXR offers “no detectable latency difference” compared to a locally rendered view (a claim we’ve heard many times before, but rarely stands up to scrutiny).

Oculus: 'We'd Eventually Like Quest PC Tethering to be Wireless'

The exact setup of the demo on stage isn’t clear at this point, so we aren’t sure if it was a demonstration of the complete data center-to-device pipeline, or a locally rendered example just showing the capabilities of CloudXR without including a networked transmission.

VR/AR Cloud Rendering, 5G, and Edge Networks

Streaming AR and VR from the cloud has long remained a technical possibility that’s  ultimately held back much more by latency than bandwidth limitations. While CloudXR is of course designed to be low latency itself, another major part of latency in the cloud rendering pipeline is the network delivery—once a frame leaves the data center, it needs to be transmitted to the headset without adding much more latency.

While 5G does theoretically have lower latency than many existing network infrastructures, so-called ‘edge computing’ is another important piece of the latency puzzle. Low latency is as much a function of the physical distance between the data center and the endpoint as it is a function of the network’s capabilities; edge computing is the concept of locating cloud data centers physically near users to reduce latency.

HTC Says '5G Hub' Will Stream VR 'Directly to Vive Headsets', One Day... Probably

A single data center in the middle of the continental US, for instance, may have too much latency by the time it reaches the country’s coasts for a viable CloudXR experience. Edge computing proposes using a distributed cluster of data centers, allowing rendering to happen at the data center nearest to each individual user, thereby reducing latency that results from physical distance.

It isn’t clear what kind of latency requirements Nvidia is recommending to make CloudXR viable, but being within range of an edge computing node may be as important to the equation as 5G.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Niklas Fritzell

    So quest 2 will be wireless and offer every single AAA without a pc if using this software? If made easy to use this is what will make VR mainstream perhaps?

    • Gonzax

      Not that simple, probably the future but very long-term in my opinion. Lots of places don’t even have access to 4g, let alone 5g. Big cities yeah, rest of the world, it would be a problem.

      • The Bard

        Who cares about the rest of the world? If you want technology, live in a big city. If you want nature, live in villages and small cities.

        • Mei Ling

          Ideally everyone should have access to the latest technologies regardless of your location and in the interest of fairness however your point is somewhat valid. If you want access to 5G infrastructure (at least in the early days) you should be located in a major town or city which I think is really down to demand and supply.

      • Ad

        5G is high frequency so it needs even more towers with short range. But I could definitely see this taking off in cities if they can reduce VR to a $300 dollar headset and a cloud subscription to play high end VR games.

  • Ad

    If they can pull this off then I could see Google or another big player buying up a VR hardware manufacturer and starting to pump out VR cloud headsets to dominate the medium. Unless NVIDIA thinks it can edge that out by offering GeForce VR through low end PCs and streaking boxes.

  • Kevin White

    Prediction: we’ll need another generational leap in wireless technology (6G) before cloud / edge for AR and VR really breaks through and becomes technically adept enough to achieve ubiquity. Set your sights on 2030. :o)

    • Andrew Jakobs

      Problem with 5G is already that it requires way WAY more power and way WAY more transmitters than 4G, it’s actually ridiculous they pushed 5G instead of trying to get to a technology that at least requires the same ammount of transmitters as 4G..

      • I know physics, RF attenuation, and the speed of light are all bitches… :-)

        • RickityRick

          I personally am a fan of the speed of light. So don’t talk smack about it, alright?

      • sebrk

        Yeah thats not how this works at all Andrew

    • bud01

      1mm is the smallest carrier wave possible hence nothing past 5g

      • Kevin White

        Nothing past 5G? 100 years from now we’ll still be using the wireless tech they’re rolling out here in quaint old 2020? That seems unlikely. Terahertz will be possible in time. In fact, in March the FCC (in the US) opened up those tremendously high frequency bands for experimentation.

        • bud01

          There is other stuff going on independent of light speed limitations. Ref ttsa high bandwidth quantum Entanglement.

    • Larry

      Nvidia and Intel would love this model to be viable because they would sell billions in silicon. Putting aside the unrealistic 5G latency numbers (actual 5G latency is 20-30ms in the real world), the edge rendering model is completely contrary to how data centers are built today. Data centers are centrally located, with cheap land and power. No company in their right mind will invest the billions needed for thousands of local data centers doing edge rendering for a few million VR users.

      One sure sign of an industry in tough times like VR is the desire to point to some new innovation as the silver bullet needed to cure all the problems. 5G is exactly that imaginary silver bullet for the lack of consumer adoption of VR.

  • bud01

    5G is going to be a game changer and really perfect for the progress of technology,

    Request Jensen Huang you get some silver EMF shielding inside that nice leather jacket to stop the biologically active effects thou.

    Any questions go and have a cup of tea with Martin L. Pall, professor Emeritus of Biochemistry and Basic Medical Sciences at Washington State University

    Buy more medical stocks….

  • I mean, there’s no real need to stream VR. AR, sure. But downloading VR content doesn’t appear to be the barrier to entry for anyone. And locally rendered content allows for that sweet sub-11ms performance.

    I guess lower spec headsets, with higher fidelity solutions perhaps. Ok, forget everything I said, bring on 5G and CloudXR

  • I know someone that has tried it and said that this SDK is still rough and needs a lot of debugging