We can see that effect in this video, which was shot through a see-through head-mounted display with a high-speed camera and is being played back at one-tenth speed.

Here, the camera is panning across a wall that contains several markers used for optical tracking, and a virtual image is superimposed on each marker – the real-world markers are dimly visible as patterns of black-and-white squares through the virtual images.

It’s easy to see that because the pixels are only updated once per displayed frame, they slide relative to the markers for a full displayed frame time (about 10 camera frames), then jump back to the correct position. Exactly the same thing happens with a full-persistence head-mounted display when you turn your head while fixating or when you track a moving virtual object.

Because this is being played back in slow motion, we can see the images clearly as they move during the course of each displayed frame. At real-world speeds, though, the pixel movement is fast enough to smear across the retina, which makes the image blurry.

MAbrash GDC2013 (38)

To give you an idea of what the smear part of judder looks like, here’s a simulation of it. The image on the left is with the head not moving, and the image on the right is with a leisurely 120 degrees per second head turn rate. On a 60 hz full-persistence display, that results in two degrees of smearing across the retina per frame – and on a head mounted display that’s 1280 pixels and 40 degrees wide, that’s a full 64 pixels of smear, which as you can see reduces detail considerably.

Also, at real-world speeds the jumping back to the correct position at the start of each displayed frame makes images strobe – that is, it causes the eyes to see multiple simultaneous copies of each image, because they can’t fuse detailed images that move more than about five or ten arc-minutes between frames.

The net result of the smearing and strobing is a loss of detail and smoothness, looking much like motion blur, whenever the eyes move relative to the display. You might think that wouldn’t matter all that much because your eyes are moving, but again, if you fixate on an object and turn your head, you can see perfectly clearly even though your eyes are moving rapidly relative to the display, and judder will be immediately noticeable.

In contrast, observe how smooth the panning across text on a monitor is here.

This nicely illustrates how head mounted displays introduce a new set of unique artifacts.

The ideal way to eliminate judder is by having a high enough frame rate so that the eyes can’t tell the difference from the real world; 1,000-2,000 frames per second would probably do the trick.

See All GDC 2013 News

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. See here for more information.


  • WormSlayer

    My brain hurts! Still I’m excited that guys like him are excited :D

  • Patrick Hogenboom

    Thanks a bundle for transcribing the whole thing :)

  • shole

    If you want to view the two videos in the presentation and don’t have ms powerpoint, you can rename the .pptx file to .zip and the files are in there as .wmv under \ppt\media\

    • Michael Abrash

      Shole,

      Thank you very much for posting this!

      –Michael

  • Andrés

    Thanks so much for transcribing this!

  • Andreas Aronsson

    I thought I would not get to read/hear the actual talk in some time :D Very interesting, somewhat alarming, but at the end uplifting and inspiring! Thanks for this! Also, you could if you wanted to rename the .pptx to .zip and extract the videos (media folder) and upload them unlisted to Youtube and embed them on this page :) Just a thought!

  • Andreas Aronsson

    Ah, in the time it took me to read this (left the article half-read over night) shole already pointed this out :x oops.

  • Esse

    “That means that if you fixate on something while you turn your head, your eyes remain fixed with respect to the real world, but move very quickly relative to the display”

    Hum, no. If your eyes are fixed (in the head referential), they are fixed in the display referential. Because the display & the head are fixed one to the other.

    This is over complicating the subject. Only matters the head movement. Not the eyes movements.

    • Ben

      I think that, in this context, “fixate on something” means a visual fixation on an object in the external world rather than having the eyes be “fixed” relative to the head. If the head is turning while you fixate an external object, the eyes must counter-rotate in order to maintain a stable fixation (eg: via the VOR). Since the VR display is attached to your head frame, this means that your eyes are rotating relative to the display, and that’s the source of the large relative motion that causes issues.

      • Michael Abrash

        Ben,

        You’re correct – that’s what I meant. If you look at a key on the keyboard and keep doing that while you turn your head, you can easily get your eyes moving at several hundred degrees per second relative to the display.

        –Michael

  • Esse

    “The human perceptual system has evolved to be very effective at detecting such anomalies, because anomalies might be thinking about eating you, or might be tasty.”

    It is not some specific “anomalies detection system”, rather that the human brain has in-depth “routines” to analyse the space & movement, and when you broke the rules the routines fail.

    Like when you are sick in a car.

    So I guess the reaction will not be fear or drooling, more puking.

  • Ben Humberston

    Thanks for the transcript for those of us who couldn’t be there!

  • Mattso

    Yeah, just echoing the sentiments above that having it all transcribed meant I actually got to digest it really quickly. Many thanks!

    Very informative stuff – and here was me thinking it was gonna be easy. :)