New instructional videos, supposedly leaked from a HoloLens development program, give us a glimpse at what every day interaction with Windows running in Augmented Reality looks like.

Microsoft first unveiled their augmented reality headset HoloLens at a special Windows 10 event last year. Since then, we’ve seen some impressive demonstrations of the technology, including live stage examples of how augmented reality technology might one day change our lives.

But, if Microsoft ever decides to sell HoloLens to the public, how will users navigate and interact with Windows 10 Holographic, the AR focused flavour of Microsoft’s operating system designed for HoloLens?

These leaked tutorial videos are for HoloLens application Actiongram, a recently revealed drag and drop holographic ‘sticker’ app, which lets you drop 3D models into your real world surroundings. Whilst the app itself is intriguingly useless, the videos walk users through the basics of how to launch Actiongram from the Windows Holographic equivalent of a start menu and generally interact with the user interface.

It’s the most we’ve yet seen of Windows Holographic in action, running real-time inside an augmented reality session and it gives you a glimpse at how Microsoft envisions human interaction with Windows working in lieu of a keyboard or mouse.

The HoloLens’ primary interaction method is via human hand gestures, interpreted by the headset. Microsoft have had to invent a new language in order for users to interact with Windows with just their hands, and you can see (and hear reference to) a few in the videos. But, if you’re wondering what “Bloom to the shell” and “Airtap” mean, Microsoft’s Windows Holographic instruction pages are now online.

Quest 'Augments' Feature for Concurrent AR Apps Needs More Time to Cook, Says Meta CTO

According to the documentation, a Bloom gesture requires a user to “hold out your hand, palm up, with your fingertips together. Then open your hand.” The gesture can be seen in part in the image below.


The Airtap gesture has had the most airing in public demonstrations and is mostly self explanatory. The user extends their index finger and thumb in front of them and brings their index finger together (kind of like the ‘pinch’ gesture on mobile phones) – this seems to be most analogous to the left mouse click action in standard windows. The gesture can then be extended by holding the thumb and forefinger together and moving your hand – useful for dragging objects, as demonstrated in the video below.

Other gesture such as a prolonged extension of the users index finger to bring up object specific, context menus – as seen in the above video where the user resizes the ‘hologram’.

These gestures and actions are combined with the user’s gaze to act as a mouse point in certain scenarios. For example, when launching an application initially, you’re required by Windows Holographic to ‘place’ the application somewhere relative to your physical space. This can seemingly be anywhere; on walls or ceilings or even furniture.

Also watch in the above videos for what Microsoft calls the ‘Holobar’, a context sensitive set of controls for the 2D representation of HoloLens apps which allows you to control relative location, shape and size of the ‘window’.


The subject of how we’re expected to interact with Augmented Realities is almost as interesting as how we display them and seemingly just as challenging. This is early days for Microsoft’s HoloLens, but it’s interesting to watch it and companies like Meta, begin to define the language of interactivity for this new form of interface for computing.

PC VR Exclusive Puzzle Adventure 'Firmament' Coming to PSVR 2 Later This Year

The ‘Development Edition’ of Microsoft’s HoloLens is to be made available to those who sign up to the HoloLens development program and are willing to lay down a cool $3000 for the privilege. These units are due to begin shipping March 30th.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Based in the UK, Paul has been immersed in interactive entertainment for the best part of 27 years and has followed advances in gaming with a passionate fervour. His obsession with graphical fidelity over the years has had him branded a ‘graphics whore’ (which he views as the highest compliment) more than once and he holds a particular candle for the dream of the ultimate immersive gaming experience. Having followed and been disappointed by the original VR explosion of the 90s, he then founded to follow the new and exciting prospect of the rebirth of VR in products like the Oculus Rift. Paul joined forces with Ben to help build the new Road to VR in preparation for what he sees as VR’s coming of age over the next few years.
  • OhYeah!

    I love how old ideas just keep rolling back around. I will soon have my old eSheep walking around on my actual desktop! (also remember the orange that would smoke and wander around?)

    • Mitroll

      Bonsai Buddy

      • garrilla

        Peddy Parrot.

  • yag

    Yet more marketing bullshit from Microsoft…

  • Mateusz

    3k? I thought Meta DK was expensive…