With Apple’s increasing interest in VR on Mac, the company is adding essential features to its new Metal 2 graphics API to support high performance VR on MacOS.

Because VR demands high-powered graphics and extremely low latency, specially tuning the rendering pipeline between the GPU and the headset is crucial to maintaining a comfortable and performant VR experience. Many of the key tweaks to the VR rendering pipeline have been established on other platforms, and Apple is bringing many of them to the updated Metal 2 API which the company says now “provides powerful and specialized support for Virtual Reality (VR) rendering.”

SEE ALSO
Apple and Valve Have Worked Together for Nearly a Year to Bring VR to MacOS

At the ‘VR with Metal 2’ session during Apple’s WWDC 2017 conference last month, Apple GPU Software Team member Rav Dhiraj detailed the new features in Metal 2 for VR.

Single-pass Stereo

Since VR headsets show a 3D image, they need to render an individual view for each eye; you might think this would require double the rendering work, but with some smart rendering techniques you can be much more efficient. Single-pass stereo allows the GPU to render to the left and right eye with a single draw call instead of one for each eye.

Metal 2 now makes this possible with what Apple calls the Viewport Array feature which allows “per-primitive viewport selection in the vertex shader.” Instead of rendering the left eye image and then the right eye image, developers define a texture that’s the resolution of both eyes next to each other, then render across the entire texture in one pass. Telling the renderer which eye each primitive belongs to ensures that it gets rendered in the correct eye-region of the texture and with the correct offset to account for the slightly different perspective of each (due to the distance between the eyes).

Direct-to-display

Direct-to-display (also known as Direct-mode) allows the GPU to more directly access the VR headset by bypassing parts of the pipeline that would be necessary only for traditional monitors.

Without Direct-to-display, the operating system would see a VR headset like any other monitor and simply mirror or extend the operating system view onto it, including modifying the rendered image with any OS-induced post-processing. With Direct-to-display, the operating system doesn’t identify the headset as a monitor, preserving the existing window layout and display arrangement. This allows a VR compositor (which warps the initially rendered image and prepares it for display on headset) to directly present the rendered image to the headset’s display without the OS introducing any additional latency, performance hits, or image tampering.

Not only does Direct-to-display reduce the complexity of the VR rendering pipeline, it also makes the end-user experience better because developers and users don’t need to fuss with ‘where’ on the OS’s desktop the VR application is displayed—instead it automatically ensures the rendered image lines up perfectly with the VR headset’s display and without any interference from the OS.

– – — – –

Metal 2 will debut with MacOS version ‘High Sierra’ which is presently in beta for developers and due to launch later this year, supporting all Macs compatible with the present ‘Sierra’ version.

On stage at WWDC 2017, Valve’s Nat Brown overviews the inner workings of SteamVR rendering on MacOS | Photo courtesy Apple

Developers building VR apps with tools like Unreal Engine or Unity won’t need to worry about the above features as the MacOS VR integrations of those engines will take advantage of them by default. Developers wanting to building native VR applications or integrate with existing proprietary game/rendering engines on MacOS will want to specifically adapt support for these new VR specific features.

Both features and more are covered in detail during the WWDC session, including an overview of best-practices for VR rendering that will be useful to developers building VR apps from scratch.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Nika

    in before apple says they invented VR

    • Get Schwifty!

      Idiot.

      • Jimmy Arias

        The correct spell is “iDiot”

    • Adrian Meredith

      iReality

  • Daemon Hunt

    *holds breath* Metal 2 better be a reality (as opposed to a virtual one… haha see what i did there?). Metal 1 was vapourware. My D700 2013 Mac Pro spec’d to the max can barely handle Second Life at medium settings lol

    • NooYawker

      Maybe they’ll finally put in a real gpu.

  • Adrian Meredith

    They could have just added support for vulkan but no they’re just shoving more proprietary crap on people

  • Lucidfeuer

    If there are things coming specifically for the iPhone 8 and further with a mobile headset, then this’ll be interesting. Otherwise Apple is so behind and will stay because of their ridiculous pricing, proprietary lagging platform, that there is no point in switching back.

    I actually switched from Mac back to PC because of 3D, CAD and VR and there was no other way around. Their ridiculous iMac pricing and the horrendous lackluster Metal are not going to change any of that. In fact pushing for Metal instead of Vulkan is probably a terrible idea.