Q: It seems the controller suffers from line of sight, can you give details about this? Ie. what is the impact on the accuracy with a hand, arm or torso between the controller and the base stations?

A: The controller needs line of sight to the sensors just like optical systems do. Well, not just like optical systems do. “Occlusion” appears to mean a different thing to sound than it does to light.

In an optical system as soon as anything comes between the transmitter and the sensor you’re done. There’s just no signal.

For sound it’s not that simple. If, say, the edge of your arm comes between things the sound still gets to the sensor. The distance it has to travel is slightly longer so the calculated (x,y,z) distorts a smidge but in game play this often passes unnoticed.

If you put a round object in the way the sound will pass around the object, rejoin on the other side and continue on its way. Again, the distance traveled will distort the calculated (x,y,z) a bit but it doesn’t affect game play all that much. Put your whole body in the way, of course, and things stop working. Of course, put a body between you and the television and you won’t do so well either.

In console game play the player is facing the television with both hands in front. The game play is more about accurate pointing and moving in 3D space rather than standing and waving your arms around. It’s not about standing there with your arms against your chest. It’s about getting your arms out there and using the room space.

Because of this, occlusions aren’t much of a problem in console play which is what I designed this system for.

Remember, that this is just the first prototype. I built it to show that the absolute positioning tracking worked at all and that it could be used in game play. It’s a proof-of-concept solution and it’s not yet an optimized one. There are a lot of things that can be done to reduce and/or remove the occlusion problem and I’ll be implementing some of those in the final version.

Q: Does the controller need to be directed toward the base stations to work or can it be oriented in any arbitrary direction?

A: I’ve worked up a number of ultrasound transmitter designs and they all have their pluses and minuses. The ‘Frankencontroller’s’ transmitter in the Skyrim and Halo videos works in any orientation. The one in the CES and Titanfall videos is smaller and has a 180 degree field. It works as long as you’re not facing backwards.

I’ve built other transmitters and I’m still experimenting. The best one will make it into the final controller. It’s sort of like “The Ultrasonic Bachelor.”

A New Category of VR Game is Slowly Emerging Right Before Our Eyes

Q: What is the frequency of the emitted ultrasounds?

A: The prototype uses 40kHz.

Q: Does the emitted ultrasound have any negative impact on animals (dogs and cats can hear ultrasounds)? Did you conduct tests about this?

A: I have a dog, two cats, a snake, and ducks (Yes, they’re in the house. Don’t ask). No complaints so far.

Q: Do you intend to produce other controllers with a similar technology but a form factor similar to the PS Move or Razer Hydra (ie. grip, thumbstick, buttons)? Why did you decide to make a break-apart game controller? Both Sony and Nintendo, when introducing the PS Move and the Wiimote, respectively, chose a design very different from their normal controllers. Supposedly, they had a reason to do that. Their current design doesn’t seem ideal as a standard game controller, or as a motion controller.

A: Anything is possible and I’m not ruling anything out. Here’s why I chose the design I did:

Ever since motion gaming arrived there’s been this war between controller gaming and motion gaming. One motion system eliminated the controller completely. Some let you have buttons but it looks like they didn’t want you to use them. They’re very awkwad and inconvenient.

By last measurement, 91% of the internet that isn’t full of nekkid videos is full of gamers flaming back and forth about which method is lame and which one ‘rools’.

I don’t see it this way at all. Honestly, I didn’t like the way motion gaming has been done. That’s why I built this thing!

Motion is fun but sometimes I still want buttons and sticks. Maybe I want to jump around the living room today but tomorrow I mean want to slug out on the couch and button mash. Maybe I want to use buttons for some things and motions for other. I didn’t want to choose. I wanted everything available all the time.

I didn’t want to lose “traditional” controller game functions so I started with the “traditional” controller. I made the button and stick configurations similar because I still wanted to be able to play the way I always had.

Then it came time to add tracking and motion. With a “traditional” controller both hands are stuck together. What’s the point of tracking that?

We need to get our hands apart so we could have full use of motion. I still want everything, though. I still wanted to be able to use the buttons and sticks even when my hands were in motion and apart. The only thing to do was cut the controller in half.

I wanted to be able to go back and forth between “traditional” play and motion play during game play without having to look at the controller. Magnets made that easy. Get the two halves close enough and they pop together.

How to Tell if Your PC is VR Ready

At first glance it sounds like this would be awkward. If you hold your controller with one hand it’s not a comfortable thing. Remember, though, I cut the controller in half. You’re only holding half of it. Take a hacksaw to your controller and you’ll see that’s it’s not that bad. Wait—don’t do that. It might have problems when you put it back together.

I made some other form changes to make it more comfortable. I changed the shape of the bottom so it’s easier to get your fingers around. All of the heavy stuff is in the grip so that the weight of each controller half falls in the palm. You don’t have to grip it. It’s balanced so that it sits in the palm naturally.

It looks the same but there are important changes that were made to accommodate the splitting. It’s quite comfortable. Hundreds of people have tried it out and agree.

I’m not finished, of course. There are still tweaks to be tried. I want to have as many people try it out as I can before committing to a final design.

Are there other controller designs that would work better? I’d like to find out. This obviously isn’t the only way to do it. I’d love to be able to offer more than one design so that players can choose whichever design they like best. I’d like to offer two options for the left controller half. If the analog stick and the Dpad are reversed it will have more of PS feel.

You can see more here. Click on Chapter 2, The Controller.

Q: Do you think that shaping it like an inexpensive console controller will contribute to the notion among potential backers that it’s not worth the $300+ you wish to charge, despite the presumably expensive tracking technology inside?

A: This is something that I thought a lot about. One of my design goals was to add motion without taking away traditional controller aspects that everyone was used to. I thought this was especially important if I’m adding motion to a game you already play. If all the buttons are scrambled up you’re not going to like it.

The “motion vs buttons” debate is annoying. I wanted “motion and buttons” not “motion or buttons.” Coming up with a different controller shape would say, “Hey! I’m different!” but it takes away my traditional play. We’re back to “motion vs buttons” again.

So, I kept a “traditional” shape. The controller becomes a “stealth” thing, just like you said. At first glance the difference isn’t obvious. Split it apart and suddenly it does all these new things. As a designer I really like this! It’s simple, elegant and it works.

City Builder 'Tropico' Comes to Quest, Letting You Become El Presidente of Your Own Banana Republic

I’ve been playing with it for a couple of years now and using motion and buttons really works. I’m hardly a “serious” gamer. I play for fun and don’t have as much free time as I’d like. On a standard controller online play isn’t much fun because I just get slaughtered. When I aim with motion I can hold my own.

The Titanfall video is an excellent example of “hybrid” motion and button play. As a Pilot on the ground, things move quickly. Be quick or be dead. I use motion for aiming but I do best using the traditional controls for moving and jumping. I use motion aiming, weapons changes and reloading because it’s fun. If I need to turn around quickly I’ll hit the right analog stick and then go back to aiming with motion.

I call down the Titan with motion (the arm over the head and all that) because it’s fun. I can do it with buttons but it’s just so damn kuwl to call the Titan like this.

When I’m in the Titan the gameplay is slower so I use a mode where I can move around the room and aim at the same time. It’s just fun! It’s become reflex to step backwards and sidestep behind cover when someone starts shooting. I really miss it if I’m using a standard controller.

When I first wrote the movement mapping I thought it would be cute but I didn’t think it would be useful. After all, how far can you move in your living room? I was so wrong on that one.

First I had to dump the idea that everything had to be 1:1. Is your mouse 1:1? If so your mouse pad must be friggin’ huge! I made the movement ratio tunable so I could crank up the sensitivity. My motion is magnified and I can move the character further with less motion on my part. Once I did this it was just too much fun. And, if I run out of living room the analog sticks still work and always do.

Everything is working (woo hoo!) but this presents a very annoying choice. The “stealth” thing works great and I can do everything I want but it doesn’t look all sexy and stuff. If I use an entirely new controller design it will get attention but I lose all of the old controller elements I wanted to keep.

I like this design because it keeps all the old elements and adds the motion without requiring it. Will everyone understand? I have no idea.

Continue Reading on Page 4…

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Psuedonymous

    He seems dead-set on this “only use the current sample” idea without a solid reason why, same with not wanting to fuse with IMU data. Fusing the position data with IMU data using a Kalman Filter (or similar) would provide the very low latency and very high update rates of the IMU with the baseline position from the ultrasonic pulses.

    It’s impressive that he appears to have made ultrasonic-only tracking less awful (though some more formal demonstrations and measures of accuracy, precision and linearity would be nice), it just seems to be a lot of wasted effort when a more effective method already exists.

    • Zig

      On page 2 Don explains why he has not sensor fused his absolute position technology.. On same page Don also explains the four ways he evaluated his accuracy. He also has a video explaining/showing it http://www.youtube.com/watch?v=8X9RDoGZ4qk

      • Psuedonymous

        Oh I read page 2, I just don’t buy “some people might be confused” as a valid reason for actively resisting making a superior system. The only people who are even going to be aware of he difference between absolute and relative positional tracking are going to be aware of sensor fusion.

        • Zig

          IMUs don’t matter until you have the non-IMU piece optimized, ie your camera, magnetic, ultrasound, or other technology. For high VR HMD or VR input presence you will need a reference point, either a visual or some other base station, so it will have to be absolute. The jury is still out as to which technology will prevail. I would suggest Don offers an alternative path to the current mantra

  • Zig

    measured sub-millimeter(0.25mm) position accuracy on every sample using audio and a home brew system. I would encourage everyone interested in motion tracking to read this article in it’s entirety. Thank you Ben Lang for posting.

  • Rokrull

    No one should ever develop anything for VR input until they are somewhat familiar with human neurobiology.
    6DoF and super low latency is exponentially more important than absolute positional accuracy. Our proprioception isn’t accurate enough to tell exact arm positioning so close is more than good enough, but it has to be quick or else it will feel like moving underwater. Much of the performance of these input devices (as impressive as they are) are rendered irrelevant to VR implementation by the facts of human biology. Prioritizing haptic feedback is a very good idea though.

    • Zig

      Yes, science is always important but the practical side of me says one should spend time playing VR games with VR “input” or VR motion controllers as much as possible to get it right. Each sample being accurate with no averaging of 10 or 20 samples certainly reduces latency and makes it quick, You can do a lot with accurate location data including accurate calculations of 3D velocity, and acceleration. I know the Wii and Kinect developers would have loved to have it. Most likely Don will need an FPGA/ASIC before he can offer an optimized VR solution including orientation. Meanwhile, I suspect Don has played more precision motion gaming with real games (such as Titanfall) than most and has learned a lot that can be applied to VR gaming.