We can see that effect in this video, which was shot through a see-through head-mounted display with a high-speed camera and is being played back at one-tenth speed.
Here, the camera is panning across a wall that contains several markers used for optical tracking, and a virtual image is superimposed on each marker – the real-world markers are dimly visible as patterns of black-and-white squares through the virtual images.
It’s easy to see that because the pixels are only updated once per displayed frame, they slide relative to the markers for a full displayed frame time (about 10 camera frames), then jump back to the correct position. Exactly the same thing happens with a full-persistence head-mounted display when you turn your head while fixating or when you track a moving virtual object.
Because this is being played back in slow motion, we can see the images clearly as they move during the course of each displayed frame. At real-world speeds, though, the pixel movement is fast enough to smear across the retina, which makes the image blurry.
To give you an idea of what the smear part of judder looks like, here’s a simulation of it. The image on the left is with the head not moving, and the image on the right is with a leisurely 120 degrees per second head turn rate. On a 60 hz full-persistence display, that results in two degrees of smearing across the retina per frame – and on a head mounted display that’s 1280 pixels and 40 degrees wide, that’s a full 64 pixels of smear, which as you can see reduces detail considerably.
Also, at real-world speeds the jumping back to the correct position at the start of each displayed frame makes images strobe – that is, it causes the eyes to see multiple simultaneous copies of each image, because they can’t fuse detailed images that move more than about five or ten arc-minutes between frames.
The net result of the smearing and strobing is a loss of detail and smoothness, looking much like motion blur, whenever the eyes move relative to the display. You might think that wouldn’t matter all that much because your eyes are moving, but again, if you fixate on an object and turn your head, you can see perfectly clearly even though your eyes are moving rapidly relative to the display, and judder will be immediately noticeable.
In contrast, observe how smooth the panning across text on a monitor is here.
This nicely illustrates how head mounted displays introduce a new set of unique artifacts.
The ideal way to eliminate judder is by having a high enough frame rate so that the eyes can’t tell the difference from the real world; 1,000-2,000 frames per second would probably do the trick.