augmented reality piano trainer ar app idea
Photo Credit

If you’ve been following along for a bit here at Road to VR, you know I’ve got a chip on my shoulder when it comes to AR. So far we’ve seen few  useful implementations of augmented reality. The technology exists to convincingly overlay virtual imagery onto our real world, but so far there’s no ‘killer app’ or usage scenario that is driving adoption of AR. So here’s a free idea for an augmented reality app: the keyboard trainer. First come, first served!

So far AR apps have been little but ‘here’s a new way to show people ads’ or ‘here’s a crutch for dying media like physical newspapers.’ AR isn’t going to take off until there’s a good reason for people to be using it; you don’t need to be a genius to see that consumers aren’t going to adopt AR if you are pitching it as a new way to cram more ads down their throats. Did people pitch the first TVs as a way to advertise to you in the comfort of your home? Of course not! They pitched the idea of the TV as a way to bring entertainment directly to you. AR is no different — there needs to be a better pitch than ‘it can show you novel ads!’

So here’s a free idea for whoever wants to take it: an AR app that helps you learn how to play the piano or even to type. I’m not saying this isn’t the ‘killer AR app’ that augmented reality needs, but it’s definitely something useful (and something I’d pay for)!

Free AR App Idea: Keyboard Trainer

If you’re like me you spent a small portion of your childhood playing with incredibly complicated electric pianos. These were often powerful units that could record multiple tracks and had hundreds of different tones. Unfortunately the little LCD display and unintuitive menus made using these functions a serious challenge — you can easily find the thick manuals to prove it!

The other day I was nostalgically playing around with an electric keyboard. On the keyboard’s LCD panel there was a representation of all of the keys. You could enter a mode wherein the keys would light up on the screen and teach you how to play a song. It was somewhat tough to map what was shown on the little LCD to the actual keys though.

Google Glass is one example of AR glasses that could be used with the piano trainer app.

It struck me at this moment how great it would be if I had a pair of augmented reality glasses and an AR app that would overlay a virtual hand onto my keyboard in the same position where I should put my real hand. This would make it immensely simple for users to match their hands to what’s being shown on the overlay and a great way to teach them basic chords and songs. It doesn’t take much imagination to flesh out the rest — the app could also have a virtual teacher that would talk to you and show you. “This is the middle C key… this is a C major chord.” Presto! Basic keyboard skills mastered in no time.

This same concept could easily be used to teach children the proper way to type on a computer keyboard. Just put on your AR glasses, fire up the app, and follow along. “This is the home row… here is where you fingers should rest,” etc.

So I say to AR app developers, this idea is up for grabs, have at it!

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Wavefunction

    The possibilities are endless, really. How to hold a bow in archery, how to throw the perfect pitch in baseball as well as how to hit it… AR apps could theoretically do it all. Although, a keyboard would be the best place to start as it takes fewer CPU cycles to determine what to overlay on a series of alternating black and white strips than it would on a random baseball bat.

    • Ben Lang

      Agreed, a piano made sense to me because it is almost like a natural marker that should be easy to track. You could even put little stickers on all of the C keys to help if needed. Same with a computer keyboard, very easy!

  • tc

    Some people from Korea had a drum prototype at siggraph 2012 emerging technologies.
    Instead of glasses, they were projecting on the snares and cymbals.
    As a non drummer, I found it really useful.