Realities wants to transport viewers to exciting places across the globe in ways that are much more immersive than the single, flat vantage point of a 360 degree photo or video. The company is creating an efficient photogrammetry pipeline to build sometimes frighteningly realistic virtual environments that users can actually walk through.

I recently visited the Realities team to check out their very latest work. After donning a Vive Pre headset and playing around for a solid 10 or 15 minutes, I was thoroughly impressed… and I hadn’t even left the menu.

No One Said Menus in VR Have to Be Boring

Realities wants to capture exciting places around the world and allow anyone to visit. Encapsulating this sense of global teleportation, the developers have created a menu which takes the form of a detailed model of the Earth floating amidst a sea of stars and our nearby Sun. I could have clicked on one of the nodes on the globe to teleport to the corresponding scene, but I was far too fascinated with the planet itself.

realities photogrammatry virtual reality (5)

Starting at about the size of the beach ball, the Earth in front of me looked like one of those amazing NASA photos showing the entire sphere of the planet within the frame. There’s a good reason it looked that way: it was constructed using assets from NASA, including real cloud imagery and accurate elevation data for the Earth’s continents (which was exaggerated to allow users to make out the detailed terrain from their satellite vantage point). The dark side of the globe also accurately lights up with a brilliant, warm grid-like glow of city-light scattered across the planet’s surface.

htc vive realities photogammatry virtual reality (3)With the Vive’s controllers I was able to reach out and manipulate the globe by spinning it, moving it, and doing a pinch-zoom gesture to make it bigger or smaller. I grabbed the Earth and lifted it over my head to look at it from below; for the first time in my life, I truly got a sense of the way in which South America really curves down under the Earth as it reaches toward Antarctica. I’d never seen—or I suppose ‘felt’—the shape of the continent in that way before, not even when playing with one of those elementary school globes in the real world.

SEE ALSO
Hands-on: 'Batman: Arkham Shadow' Brings the Series' Signature Combat to VR

I went looking for Hawaii, where I once lived for a year. After spotting the lonesome island chain in the middle of the Pacific I wanted to zoom right down to street-level to see my stomping grounds. When I couldn’t zoom in much further than the ‘state view’, I had to remind myself that this was, after all, just the menu. At this point I realized I’d been playing around with it for 10 or 15 minutes and hadn’t even stepped inside Realities’ actual product.

realities photogrammatry virtual reality (4)

So I swung the globe around to find a highlighted node in Europe (where the Realities devs are from, and many of their early captured scenes) and clicked to teleport.

Stepping Inside a Virtually Real Virtual Environment

realities photogrammatry virtual reality (6)
This is an in-engine screenshot, not a photograph.

When the scene faded in, I found myself in a small, dilapidated auditorium. Now, this would have been a ‘neat’ place to see in the form of a high resolution 360 photo. But with Realities, the scene around me not only had depth, it was actually made of real 3D geometry and was fully navigable, both by physically walking around the Vive’s room scale space or pointing with the controller and clicking to blink from one spot to the next. There’s a massive gulf in immersion between just seeing a space like this with a 360 photo compared to being inside a detailed recreation where I can walk around and kneel down to see the dust on the floor.

htc vive realities photogammatry virtual reality (2)
Amazing how fascinating a dirty floor can be in virtual reality when done right.

For the most part, the scene around me was more detailed and realistic looking than any other virtual world I’ve been inside. This is thanks to the photogrammetric technique that Realities is using to capture these spaces, which involves the use of high resolution photography to recreate a very accurate model of the real space.

SEE ALSO
'Mobile Suit Gundam: Silver Phantom VR' Quest Hands-on: Promises of Immersion Dulled By Restrictive Gameplay

The developers joked that they get to “outsource the lightning engine to the real world,” since the scene is lit as desired in real life before capture. With photogrammetry, the real world also handles the jobs of texture artist and 3D modeler (with a little help from someone like Realities who has to find a way to pull it all into the computer).

The results are only as good as the pipeline though; Realities have made big improvements since the last time we saw their work. Inside the crumbling auditorium I could see the debris scattered across the room with detail that would be simply unfathomable to craft by hand in a 3D modeling program in the time it took Realities to capture the space.

Even the most minute details were captured and shown as fully realized geometry. As I approached the walls I could see the flecks of paint curling and flaking off after decades of neglect. The graffiti-covered walls revealed individual strokes of paint up close, and scattered about the debris I spotted the discarded yellow cap of a can of spray paint. The developers told me people often find tiny details—like a pack of cigarettes and a lighter—that they themselves haven’t spotted before. No joke… a scavenger hunt in a space like this would probably be a lot of fun.

View post on imgur.com

An older capture of the famous Las Vegas sign

In some areas where adequate data wasn’t captured (like in tight nooks where it’s hard to fit a camera) the geometry will look sort of like a janky 3D capture form the original Kinect, with stretched textures and unearthly geometry, especially on very fine edges. Some of this is hidden by the fact that real space I was looking at is quite derelict, so it’s already a chaotic mess in the first place. Still, the Realities team has shown that wherever they can capture adequate data, they are capable of spitting out richly detailed geometry on both the macro and micro scales.

SEE ALSO
Sequel to Quest's Most Popular VR Boxing Game Teases Release in New Trailer
budget-cuts-(4)
See Also: 7 Ways to Move Users Around in VR Without Making Them Sick

I should be a bit more clear about the locomotion method employed by Realities. It isn’t a ‘blink’ as much as a ‘zoom’. When you point and click, you traverse from A to B almost, but not quite, instantly. You still see the world move around you, but it’s very fast. So fast, the developers tell me, it’s beyond normal human reaction time. So while you still see it, by the time you can react to it, you have reached your destination. That’s the hypothesis at least, and in my testing it seems to have the benefits of nausea-free blinking, but helps you keep you better oriented between where you are and where you came from, instead of just teleporting instantly from A to B.

Continue Reading on Page 2…

1
2
Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Dr_I_Needtob_Athe

    Man, it’s pure torture waiting for my HTC Vive.

    • Muddy

      Hell yes!

    • realtrisk

      Couldn’t agree more! Time hasn’t gone this slow for me since I was a kid waiting for Christmas!

    • Peter

      – New pc; check
      – Room cleared: check.
      – Several days off from work after release; check.
      -……

      lets…lets browse the web some more for vive news and youtube clips…

  • Haczar Criollo

    Can’t wait for hidden object games that use this tech. Great job by the Realities team

    • Some Guy

      Perfect timing for Easter…..Lets do some VR Easter Egg Hunting

  • Jeremy Swanson

    I gotta wonder what this tech could do for crime scenes. How helpful would it be for a homicide detective to be able to go back to the pristine crime scene in VR whenever needed?

    • kr_metal

      The only issue would be virtual tampering. Less than honest workers may tamper with the virtual crime scene to frame or divert attention away from a suspect. There probably will be a law that when used for law enforcement, they have to use untampered info.

      • boniek

        While certainly possible digital signature of scene files could prevent this after creation.

        • brandon9271

          Exactly! They could just use something basic like MD5 checksum and put the file on a disc inside of a safe… but honestly it not like crooked cops couldn’t tamper with REAL evidence already. I don’t see this as an issue.

      • Jeremy Swanson

        How does this risk differ significantly from the photos they currently take?

      • Median N. Mean

        Police still use dioramas to train investigators. This would oefinitely be an improvement.

      • Infiniteworld

        Have you read The Great North Road?
        Highly recommend :)

    • Rufus

      I watched a short video series on YouTube about this very thing. They used a drone to fly over a crime scene and photograph everything so they could create a 3D model and analyze it.

  • Kai2591

    Simply awesome :)
    I started to imagine a fantasy mmorpg with this kind of REAL life graphics.
    That would be either extremely cool or extremely bizarre/surreal (uncanny valleys everywheree~)

  • Tako

    Pretty sure the devs joked about outsourcing the ‘lighting’ engine, and not the ‘lightning’ engine, although the latter would be pretty awesome too…

  • Oliver Markowski

    Since when is this technology so special? Everybody is using it since ages and Agisoft Photoscan is an absolutely amazing and fully featured tool to do that kind of stuff.

  • marko

    Getting the photogrammetry in the computer is one thing.

    Getting it out is another.

    I saw a Euclideon demonstration at the World Science Festival in Brisbane last week which shows real-time ‘unlimited detail’ virtual reality rendering of 3D photogrammetry captures.

    • Infiniteworld

      I thought those guys were totally vapourware. Are they really making stuff? Their demo videos from a few years ago logged amazing

  • Simon Che de Boer

    https://youtu.be/dzucXijIbPE our example. We use normals and dynamic lighting.

  • PaulDean

    Please excuse me and feel free to have a laugh or 2 at my expense but I
    am only an egg. I have never been a gamer but am VERY INTERESTED in
    acquiring your Inside Realities program for my HTC Vive VR. I am also
    looking for educational titles as it were as in Museum Tours, Ocean and
    Space Exploration, etc. but am more interested in the Inside Realities.
    Can you tell me where I might get it? Any/all help is GREATLY
    appreciated.

  • Scott Barrett

    I would love a death match in the oval office, providing Trump could be an executable NPC. Um… did I just get myself put on some FBI list?

  • Very nice one. Second best after amazing Auckland VR realityvirtual.co experience “MANA VR”. Our team finished now a very real virtual reality development on Android in which you can clearly walk (no flashing or switching!) by using your bluetooth controller and if you don´t have that, you can simply enjoy the guided tour. Find your proper Samsung S7 download here: https://play.google.com/store/apps/details?id=com.BW.lsVR&hl=de