In an interesting case of software pushing the limits of hardware, Valve recently updated SteamVR Tracking code to account for the speed of high-level Beat Saber players.

At first glance it might not be apparent, but Beat Saber is an excellent challenge for a VR controller tracking system. You might think that a fast-paced VR shooter or sword fighting game would be the game that would be pushing SteamVR Tracking to its limits, but Beat Saber is actually far more demanding.

In a recent update to SteamVR Beta (which will soon roll out to the SteamVR main branch), the following sentence appeared in the patchnotes:

Increase limits of what we thought was humanly possible for controller motion based on tracking data from Beat Saber experts.

Though it seemed like it might have been just a joke, Valve developer Ben Jackson offered up some detail in the comments:

The tracking system has internal sanity checks to identify when things go wrong. For example, if our math says you are behind your only basestation, clearly we made a mistake, because we wouldn’t be getting any signal from behind the basestation. One of these checks relates to how fast we thought it was physically possible for someone to turn their wrist. It turns out that a properly motivated human using a light enough controller could go faster (3600 degrees/sec!) than we thought.

Essentially, Valve built a number of assumptions into the SteamVR Tracking code so that it can identify clearly erroneous data (like a controller signaling that it is moving way faster than should be possible). However, it seems that one of those assumptions was actually too low for the speed of some high-level Beat Saber players, and Valve has now adjusted the code accordingly.

The main reason why Beat Saber manages to push the limits of VR controller tracking so effectively is not just because of the top speeof a player’s hands, but because of the drastic changes in acceleration when players are whipping their hands back and forth to cut distant notes on time.

SEE ALSO
Hit Viral Song POP/STARS Comes to 'Beat Saber' Free on All Platforms

Most VR controller tracking systems use two tracking methods and combine them together. The first system is an IMU inside each controller which senses rotation and translation, and can do so with low low latency and high frequency. The issue however is that IMU’s are prone to ‘drift’, so they can’t be relied upon alone for tracking. That’s where the other tracking method comes in: in the case of SteamVR Tracking, that’s the external ‘basestations’ which are used to establish the absolute position and rotation of the tracked object and to correct for drift from the IMU. However, the basestations update slowly compared to how fast the IMUs are (in the ballpark of 100Hz vs. 1,000Hz), and until that next positional correction comes in, all the tracking is up to the IMU.

In many cases, the moments of tracking between position corrections is aided by prediction, which uses prior motion to project where the tracked object is heading next. When moving slowly or drawing predictable arcs through the air, this works well. But Beat Saber is something of a worse-case scenario for that prediction because there’s no way for the tracking system to predict when the player is about to suddenly and completely reverse the direction of their controller (which happens constantly in the game, especially at higher levels).

Despite the challenge, SteamVR Tracking has held up extremely well to Beat Saber’s tracking needs except, it seems, in some edge cases which have now been patched. It’s frankly pretty amazing that the system manages to work as well as it does for the game; Valve ought to be pretty happy that they chose to aim for such a high tracking performance threshold from the outset.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • LOOOOL

  • Pizzy

    This why you buy a Vive people for the tracking lol. Oculus really sucks in that department.

    • xxTheGoDxx

      Oculus tracking with 3 sensors and no USB issues is virtually just as good as the Vive, as long as your play space isn’t significantly larger than 3.5 by 3.5 meters. You actually have less occlusion with a Oculus 3 sensor setup compared to a Vive Steam VR 1.0 setup (even though the latter’s two Lighthouses have a bit more FOV each than an Oculus cam) and you don’t have the slight wobble that some Vive users seem to notice when standing still due to the moving parts in at least 1.0 Lighthouses.

      Oculus tracking used to be hot garbage when Touch first came out, but was fixed completely four months later.

      • Pizzy

        Whatever man. I have never had wobble. I use two vive in my big play space. I also use tpcast have no issues. You list the short comes of Oculus camera based tracking yourself. I also don’t need to take up a ton of usb ports and run wires, just power. If you go through a pros and cons list vive is def better no doubt. People who understand the tech know this and that’s why hella people use/buy vive.

        • Proof XR Lab

          I’ve owned number of Vives and Rifts as well as using commercial systems like Optitrack.

          Vive is much easier to setup for roomscale and can scale to a larger space, see my image for a space i got working with 9 metre diagonal gap between basestations.

          However, Oculus has been measured as actually being slightly more accurate in terms of sub millimetre precision with less “jitter”, its easy to test with headset mounted stationary in playspace.

          In practical use though there is no noticeable difference to the user, both Constellation and Lighthouse are excellent tracking systems.

          https://uploads.disquscdn.com/images/8d28cad36096ba94ccfd78040b8b2375cad9e1774f31eff580ca1032f802012f.jpg

          • Pizzy

            Well I would still love to see some top beat saber expert+ players compare both. I have owned both also, but I sold my Oculus’s early on. Glad to hear you say they are now fixed etc. Can agree they are both better than Window MR gear tracking lol? I’m looking forward to Quest more nonpress reviews.

          • Proof XR Lab

            It would be interesting to compare! The Touch has slightly lower mass (160gm with battery, Vive wand is 200gm) which can be beneficial when making rapid movements.

            Both Vive and Rift use their IMU for high speed movement tracking, typically sampling at 1000hz but reporting at 500hz. Drift correction on Lighthouse at 100hz and Constellation at 60hz.

            WMR headset tracking is very impressive but controller tracking has noticeable limitations both for velocity and field of view. I recently played Beat Saber on my WMR (Lenovo) and it held up pretty well, but a “bow” game like “In Death” not well at all. https://uploads.disquscdn.com/images/e10aba3974218e5e4221beb7e2254e2c4408af45fb46c288a5ce2eae1ec6cdc2.jpg

          • benz145

            @disqus_ll11R4OGvv:disqus I really appreciate all the quality comments and discussions you’ve been sharing comments : )

            Are you on Twitter?

        • Lulu Vi Britannia

          Comparing RECEPTORS (cameras) to EMITTORS (the lighthouses). Typical behaviour of someone who doesn’t actually understand what they’re talking about…

          Did you know? The receptors the Vive use are the EXACT SAME as the Oculus Sensors.
          But hey, they say “laser” so obviously it is high-tech!!!

          Jeez, why can’t people just accept that they have limited knowledge? That way they would say much less bullcrap.

          • Caven

            No, they’re not the same. Oculus uses uses IR cameras which provide a 2D matrix of data, whereas Vive sensors are just IR light sensors. Oculus sensors are used to directly obtain 2D positional data which is used to calculate 3D position, whereas Vive sensors are used to directly obtain timing data, which is used to calculate 3D position. The only thing they have in common is the ability to detect IR light. The amount of data and how it’s used is completely different.

          • Proof XR Lab

            A great source for technical information on SteamVR (lighthouse tracking) is Triad Semiconductor; you can also purchase kits and components to build your own tracked objects, controllers, headsets. Shown is an image i grabbed whilst looking through their resource using ChromeVR in a Daydream headset.

            https://uploads.disquscdn.com/images/70a593c8bc0c7d8647d3f698efe77cf2635f0b44188a8eafbf0ef8d725d3ce00.jpg

          • Pizzy

            lol ask Alan the dev of lighthouse tech man, they are not the same. Lasers are in fact better than the cams in this instance. Oculus sensors which are cameras are limited by the glass etc.

          • Jistuce

            He’s not wrong that you’re comparing apples and oranges. The IR laser array of the Vive replaces the IR lED array on the Rift, not the cameras. And the IR sensors on the Vive replace the IR cameras on the Rift.

            The SteamVR setup is technically superior, but also far more costly to implement, mostly due to the mechanical complexity of the lighthouse assemblies.

          • MasterElwood

            LH tracking is NOT superior. It has more problem with reflecting surfaces like windows or mirrors, more jitter – and 3 or 4 sensors are also way better for occlusion.

            “laser-tracking” sounds really fancy – but it has many flaws (that’s why we have LH2).

          • Jistuce

            Vive has WAY more than four sensors.

          • Proof XR Lab

            Vive Pro uses TS4231 (Triad 2nd generation Light-to-Digital Converter IC). 32 sensors on headset if i recall correctly?

            Haven’t pulled a Pro apart yet, but this image shows an original Vive with the 1st generation Triad sensors. https://uploads.disquscdn.com/images/7be09224353e908f76d245900b30de428e2414ce5aa7bf843212c486dff4ce70.jpg

          • MasterElwood

            I am talking about the external RIFT sensors – not the sensors on the HMDS.

            4 sensors in every corner vs. just 2 base stations in 2 corners.

          • G-man

            4 sensors in every corner? you either have a lot of camera or not many corners. how does a room with one corner work?

          • Caven

            I haven’t had that problem, even though the room I use for VR has about 10 square meters of reflective glass surfaces.

          • G-man

            not really, rift has the same issues with reflections if it seesm ir leds in those reflections. vive has no jitter is you mount the lighthouses properly. and the lighthouse only needs to hit two sensors on a tracked obect for that object to then triangulate its position accurately. so occlusion wise, it depends on what type of occlusion, youd have to be doing something very wierd to coopletely cover a controller from being detected by two ligthhouses.

            abd yeah we have 2.0 because there are even more superior ways the tech can be implemented. you cant go anywhere else with cameras.

          • jj

            @luluvibritannia:disqus its funny you made this comment because you were quickly proven wrong right after you said people need to accept that they have limits knowledge

        • Andrew Jakobs

          oh man, you are really biased.. If statistics on steam mean anything, the Oculus Rift is still the boss, so your claim of the amount of people buying the vive is BS. And people who know the tech also know that they are just 2 different types of tracking, both having their pro’s and cons.. Personally I don’t care, the advantage of the Vive is that it takes less resources (usb ports etc) than the Rift. But then again, the Vive is still a lot more expensive as the Rift, and the controllers aren’t the best there are (I hear a lot of people having to have replaced their vive controller a couple of times, which isn’t a cheap thing to do).

          Again, just buy the one you’re comfortable with and just play, don’t get to hung up on the tech, the experiences in the end are about the same (especially if you don’t have a side by side comparison).

          • NooYawker

            If room scale is what you’re after then the Vive is not a lot more expensive. After you purchase a third sensor and cables you are pretty close in price.

            If seated VR gaming is what you’re after then you save $100 with an oculus.

          • Andrew Jakobs

            uhh.. here the difference between the htc vive (600-650 euro’s) and the oculus rift(399 euro’s) is 200-250 euro’s.. an extra sensor is 69 euro’s, and the cables, well, you can get them for pretty cheap, so still more than 100 euro difference for roomscale.. If it’s even seated you’ll save 200-250 euro’s.. That’s almost the price of a GTX1060/6GB..

          • NooYawker

            I don’t know where you shop but the difference in price in the US is $100-150.

          • Andrew Jakobs

            Vive store: HTC vive (including shipping and tax) 599 euro’s
            Oculus store: Oculus Rift (including shipping and tax) 399 euro’s
            There is no other way to buy it for less here in europe (except when it’s on sale somewhere, but I’ve never seen them cheaper here than on the official stores), don’t get me started on WMR headsets, which are very cheap in the US, but here, mostly still the same price as when they were introduced. And sadly the Samsung Odyssey(+) isn’t even available here..
            So count yourself lucky to be able to buy it for such low prices.

      • insum snoy

        No it isnt. I own both devices and the Rift with 3 sensors being better or as good as Vive tracking is an outright lie. And there are literally like 10 people at most posted videos of tracking wobble out of the hundreds of thousands that own a Vive and never had it.

    • Andrew Jakobs

      And yet the best beatsabre players are using Oculus, funny… I guess you’re pretty biased and have no idea about the current state of Oculus tracking. Hell yeah, the default fullbody roomscale tracking out of the box isn’t the best due to only 2 camera’s, but add one extra camera (which still is even cheaper than the regular Vive set) and that problem is also fixed, for standing 180 degrees tracking (like most games use) the 2 camera’s are very good.

      But who cares which one is better, if it works for you, stay with the one you like.

    • NextWorld VR

      Clearly you have never used a RIFT, Our upper speed never needed to be turned up, it already was! I have a perfect coverage 4 sensor setup, 35 feet either way. Put a heavy uncomfortable Vive to shame!

    • psycho_t1

      Sorry, but having just come from a day 1 Rift to a brand new Index, while the tracking for slow, basic movement is very consistent with Lighthouse, it’s an absolute joke at high speed. My controllers drift about a foot from my IRL position with fast movements.

      I’m only reading this thread because I’m searching the net for some kind of solution before I return my Index. Frankly the best feedback I’ve gotten is that Oculus’s Constellation tracking was just better, and I’m sadly quite sure they’re correct.

      Until you’ve tried playing something fast paced like boxing or beat saber with both systems, maybe don’t give people this kind of advice. I’m pretty annoyed that people are so ill informed about this to be honest.

  • Alan Dail

    How does that compare to the tracking of Rift and Oculus Quest?

    • Pizzy

      Quest is not out, so wait for reviews before you buy. If it is nice I would buy one.

  • Pizzy

    Expert+ Beat Saber players I wanna see them do a Oculus to Vive comparison on the tracking, head to head. If I had to guess all top beat saber people use vive. Also you gotta love Valve for looking at data and listening to players and making code changes. You hear Oculus doing that?? Also doesn’t take a genius to see that valve has been focusing on steamvr way more recently we can only hope this means they plan to release some of their own hardware soon. Valve is smart and probably sees the uptick trend on VR growth.

    • Auros

      Out of the top 10 Beat Saber players, half of them use Oculus. Out of the top 50, 17 use Oculus. The top two players in the world use Oculus. When it comes to Beat Saber itself, Oculus controllers are lighter which is a benefit to a lot of people while Vive has better tracking which is a benefit to others. I myself use Vive so you know I’m not biased. It’s not really hardware specific tracking limitations which is the problem. This SteamVR beta update has improved my tracking so much that I was able to beat the hardest ranked song in the game (at the point). You can find a link to the global leaderboards here (made by a developer of the game). I am also sharing a video of the top player currently (AtomicX, Rift user) playing the hardest ranked song. If you watch the video and see the speed of the song, you can see that neither system has an overall benefit over the other when it comes to playing. Both are perfectly viable! Both Oculus and Vive players have been #1. I am also leaving a link to a spreadsheet called Beat Stalker, which is where the top players themselves have given information about what systems they use, as well as physical attributes like their height and weight. Hopefully, this is enough of a comparison for you.

      • benz145

        SteamVR Tracking definitely has certain benefits over Oculus Constellation. If we’re talking specifically about Beat Saber, I’m not so sure that Vive has better tracking (latency, precision, and accuracy). I’m not saying it doesn’t, I’m just not necessarily convinced that it does.

        However, my gut feeling (as a top 500 player on official Expert+ songs) is that any differences in tracking performance between the two are overshadowed by the difference that the controller makes. I usually play on Vive because I like the feel of the wands for Beat Saber, but Touch is lighter and has a more concentrated center of gravity, which means less inertia and rotational inertia for the player to deal with.

        • Proof XR Lab

          Touch and Wand put the hand and wrist in a very different pose, the ergonomics are an interesting study. Some interactions will benefit from Wand “tool” for others Touch “hand” is more beneficial? https://uploads.disquscdn.com/images/8a0f0b36a3d4131081860b7f970131243225113b1ed866b506369cb5339bee7b.png

          • benz145

            Yup, I prefer the ‘tool’ shape for Beat Saber, but the ‘natural grasp’ of Touch for lots of other content. Hopefully Knuckles will be a great combination of both.

          • Gary Keaney

            Yet when I play Beat Saber on my Rift I adopt a sword like grip, ignoring the triggers. Never used a Vive so no comment there.

        • MasterElwood

          The best tracking possible is still 4 Rift sensors all on USB 3.0. Even slightly more precision than STVR tracking – but virtually no occlusion.

      • G-man

        beat saber or any game for that matter, no game pushes either steam vr or constellation hardware to its limits based on the speed of the tracking. running at 120hz, or even at 60hz. you would have to be hitting blocks in entirely different location at 60 blocks per second before the tracking on either would start to miss your hand position changes. no one can move that fast. the only problem steamvr tracking had was it had an arbitrary limit that was far slower than the tracking system is capable of reading but in the code it set of a red flag because it seemed to fast to be humanly possble.

        imagine a scene where the flash is running around at super speed and everything is going really slowly, and the flash runs past usain bolt in full sprint. he still looks like hes moving incredibly slowly.

    • Andrew Jakobs

      Maybe you don’t hear Oculus about it because they don’t make it public, they just adjust their software and release. OR maybe it’s because Oculus doesn’t have this problem, or maybe nobody really noticed it and tested it yet. Oculus also listens to their customers, that’s apparent in their sdk/software releases.

      • NooYawker

        Do you really believe that? You think Facebook does anything quietly?

        • Jistuce

          They hoover up creepy amounts of life data quietly.

        • Andrew Jakobs

          Yeah, because they do release updates to their oculus sdk/drivers..

    • G-man

      they casused the trend in vr growth….

  • benz145

    Ah sorry I’m not in the loop on insta. Hit me up on Twitter @benz145

    • Andrew Jakobs

      Or just use email… haha..

      • benz145

        But then we’d have to post our email addresses here in public : P

        • Andrew Jakobs

          haha, you’re right, didn’t think of that, always thought even disqus had a private chat, LOL.. stupid me ;)

  • NextWorld VR

    but… there is a way for the software to know when the controller will change directions, right after coming into contact with a target! (on to the next…)

    • benz145

      You’re right, you could theoretically build additional assumptions into the prediction if you were able to draw data from the game. For most games that wouldn’t be relevant though (a rhythm game is a little easier since there’s a predictable map of what the movement ought to look like). So I can understand why they wouldn’t spend the time to develop such a system; all the tracking is handled at a much deeper level though before the information gets passed to the game.

  • Nathan Benkhe

    If people are turning their wrists 3600 degrees in a second.. as in ten full rotations per second I seriously advise against an update… let the software limit their movements! Supporting this kind of thing is just aiding people in their pursuit to give themselves a serious repetitive strain injury. Jesus, I could not imagine somebody moving their wrist that fast that quickly. That has got to be causing players some serious wrist damage!

    • Caven

      That won’t help, because people were already moving that fast despite the limits.

    • G-man

      no one is moving that fast for an entire second. the steamvr tracking runs at 120hz. so it is measuring a 3600/120 = 30 degree movement in 1/120 of a second. or if its base don the imu its 3600/1000hz, so a 3.6 degree rotationover 1/1000 of a second. but probably only for 1 or 2 thousandths of a second.