Powered by the PS4, PSVR might not be the most powerful VR platform out there, but the newly released Joshua Bell VR Experience shows that execution—not oomph—is a major part of what makes VR great. The VR video experience uses inventive techniques to give you an immersive front row seat to a world-class violinist, featuring positional tracking and impressively sharp visuals.
Having watched the immersive video landscape for VR emerge over the last several years, there’s a breakdown of type beginning to emerge. There’s ‘360 video’ (good), ‘360 3D video’ (better), and most recently ‘VR video’ (best). The latter includes all the features of the first two (360 + 3D), but with one major addition: positional tracking. That means that you can move around within some volume inside the video scene. We’ve seen this sort of thing in action from HypeVR, and can confidently say that when it comes to VR headsets, it’s the VR video experience that we ultimately want.
Sony’s newly released Joshua Bell VR Experience falls into the VR video category, and it’s among the best VR video productions you can find today on any VR headset, despite the PSVR’s somewhat lower resolution and less powerful hardware compared to PC VR headsets. That’s thanks to a carefully planned production which incorporates a number of interesting techniques to allow for much higher quality capture than standard 360 cameras, and the significantly improved immersion of positional tracking.
The techniques used may not be broadly applicable for all subject matter, but as far as sitting you in a live-action scene next to a world-class violinist and accompanying pianist, the execution is phenomenal. Even if you aren’t into classical music, there’s no excuse not to check out this free experience if you own a PSVR.
Sony isn’t sharing their production techniques in great detail, but I can take a few layman’s guesses at how it’s been done. It looks to me like Sony filmed the musicians separately, up close with somewhat wide-angle lenses (but probably not even 180 degrees), so that they could concentrate the camera’s entire resolution on the subject rather than spreading it around the rest of the scene. Then (I would wager), they used photogrammetric techniques to create a volumetric capture of the empty performance space and (separately) the objects spread throughout the room seen in the final shot. Eventually they would digitally insert the musicians and the objects into the volumetric scene as individual pieces of 3D geometry. So basically they’ve built a recreation of the scene as real 3D geometry (rather than flat video frames), which is played back in real-time, affording the the positional tracking ability.
It’s all done seamlessly (figuratively and literally, I see no stitching seams which are common in 360 degree captures), making this arguably one of the highest quality VR video pieces you can see on any VR platform today.