Efficient text entry without an actual keyboard remains an industry-wide challenge for unlocking productivity use-cases in XR headsets. Researchers have created a comprehensive catalog of existing text entry techniques to codify different methods and analyze their pros and cons. By making the catalog freely available, the researchers hope to give others a head start on creating new and improved techniques.

Guest Article by Max Di Luca

Massimiliano Di Luca leads the VR Lab at the University of Birmingham, UK, where he is an Associate Professor in the School of Psychology and in the School of Computer Science. He previously worked at Meta where he pioneered work on hand inputs and haptics for VR. His most recent collaboration with industry was recently recognized by the ACM SIGCHI 2025 awards for pioneering the interaction framework of Android XR through exemplary industry-academia collaboration, establishing foundational input methods and interaction guidelines for XR operating systems.

As immersive experiences become increasingly sophisticated, the challenge of efficient text entry remains a crucial barrier to seamless interaction in virtual and augmented reality (VR/AR). From composing emails in virtual workspaces to logging-in and socializing in the metaverse, the ability to input text efficiently is essential for the usability of all applications in extended reality (XR).

To address this challenge, my team from the VR Lab at the University of Birmingham (UK) along with researchers from the University of Copenhagen, Arizona State University, the Max Planck Institute for Intelligent Systems, Northwestern University, and Google developed the XR TEXT Trove—a comprehensive research initiative cataloging over 170 text entry techniques tailored for XR. The TEXT Trove is a structured repository of text entry techniques and a series of filters that aim at selecting and highlighting the pros and cons of the breadth of text input methods developed for XR in both academia and industry.

These techniques are categorised using a range of 32 codes, including 13 interaction attributes such as Input Device, Body Part (for input), Concurrency, and Haptic Feedback Modality, as well as 14 performance metrics like Words Per Minute (WPM) and Total Error Rate (TER). All in all, the number of techniques and extensivity of the attributes provide a comprehensive overview of the state of XR text entry techniques.

Several key takeaways can be surmised from our research. First and foremost, text input performance is inherently limited by the number of inputting elements (whether fingers, controllers, or other character selectors). Only multi-finger typing can lead to performance comparable to touch-typing speed with a keyboard on regular PCs. As visualized in the plots below, each additional input element (or finger) adds about 5 WPM speed on top users.

Words per minute using multiple fingers, and different input devices. (each dot represents one technique analyzed in the study).

Our research also indicates that haptic feedback, the presence of external surfaces, and fingertip-only visualization are preferable ways to improve typing performance. For instance, typing on surfaces (instead of in mid-air) contributes to a more comfortable and potentially more efficient typing experience. External surfaces also minimize sustained muscle strain, making interactions more comfortable and reducing the onset of Gorilla Arm Syndrome.

Finally, and more interestingly, as of today, no alternative has fully replaced the keyboard format, probably because it still delivers the highest words-per-minute. Perhaps because it also requires high learning curves. We believe that the main path for faster typing in VR than PC might lay on the need to reduce travel distances on a multi-finger keyboard via Machine Learning and AI. XR needs its own ‘swipe typing’ moment, which made one-finger typing on smartphones much more efficient.

SEE ALSO
Meta is Experimenting with a Home Theater Environment for Quest's Horizon OS

In that regard, the deep dive from the XR Text Trove represents a significant step towards a more comprehensive understanding of text input in virtual and augmented reality. By providing a structured and searchable database, we aimed to offer a resource for researchers and developers alike, paving the way for more efficient and user-friendly text entry solutions in the immersive future.

As we explain in our paper, this work has the potential to significantly benefit the XR community: “To support XR research and design in this area, we make the database and the associated tool available on the XR TEXT Trove website. The full paper will be presented at the prestigious ACM CHI conference next month in Yokohama, Japan.

Several authors in our team are co-creators of the Locomotion Vault, which similarly catalogs VR locomotion techniques in an effort to give researchers and designers a head-start on identifying and improving various methods.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.

Road to VR
  • Christian Schildwaechter

    I REALLY appreciate the systematic approach and trying to provide not only a list of available input methods, but also metrics to actually compare their performance. I'm somewhat disappointed though not to see any of the IMHO absolutely obvious options that have been around for literally decades, often even coming directly from some of the most prominent (wearable) computing prototypes.

    Physical keys will always have benefits, but cannot be seen in VR, making finger positioning a problem, in addition to space limitations. Both issues were already covered in Engelbart's "Mother of all demos" from 1968 with a chording keyboard that uses just five keys pressed in combinations like chords on a piano to replace a regular keyboard. There have been a number of commercial versions like the Octima or the Bat for a long time, and while they require learning the chords, stenotype users reach 300 WPM by adding shorthand..

    And the (not chorded) Twiddler has been used since the earliest wearables in the 90s as a strap-on keyboard. It is still around, the 2024 Twiddler 4 is sold by Tek Gear for USD 229, even looks like a VR controller and now connects via bluetooth. A chorded keyboard could easily be integrated into any VR controller by adding a few extra buttons for chorded keying into the grip that are usually ignored and only activated while typing.
    https://uploads.disquscdn.com/images/5a2a4858deff19a2012047f1f5c331156a3dc579788d7faf4d37a7eae6dd1ed7.jpg

    • psuedonymous

      The main issue with chorded keyboards (particularly ones integrated into other controllers) is they have a very steep learning curve for even minimum viable utility, and that learning is almost entirely non-transferrable – nearly nobody is going to use a chorded keyboard controller outside of VR due to the presence of as-fast-or-faster standard keyboards being ubiquitous.

      • Christian Schildwaechter

        True. The learning curve is very steep, but also short. It's kind of a sink-or-swim situation, and people managed to get there within a couple of hours. Nonetheless chorded keyboards have remained in niches for enthusiasts willing to invest the time, or handicapped users that simply cannot use regular keyboards. This also led to these keyboards being either DIY or very expensive, as with most assistive technology.

        The non-transferrable skills also stopped typing improvements like the Dvorak keyboard layout that works just fine with any QWERTY keyboard hardware. But people don't only use their own computers, and it is very unlikely that a random machine can be easily switched to Dvorak, forcing users to regularly have to mentally switch from what has become muscle memory. And even though the QWERTY layout was created 150 years ago for mechanical type writers to avoid jamming, and is actually rather bad for todays use, it will stay, because a lot of people are familiar with it and it works both for touch typing and two finger hunt and peck typing.

        But QWERTY simply doesn't work for most of VR, because hunt and peck requires seeing the keys, the keyboard pretty much requires to be placed on a surface, and many users will have touch controllers strapped to their hand that get in the way. So you'd pretty have to learn to touch type and switch to a smaller form factor anyway, taking away the gradual transition from noob to pro that kept QWERTY alive and chording and Dvorak in the dark, so more people might be willing (or forced) to give it a try.

        And as the main use of VR is still gaming, there are a number of ways to make the steep learning curve less painful via gamification. There are already lots of flat typing games like "Epistory – Typing Chronicles" or "The typing of the dead" that make correctly typing words the main mechanic. I'm pretty sure someone could easily come up with a simple game/tutorial teaching chorded keying by first introducing a few characters and using them in the game, then gradually extending to the full character set with increasing levels. Similar to what Meta did for Quest hand tracking with "First hand" or Valve with "Aperture Deck Job" for the Steam Deck input options.

        • Massimiliano Di Luca

          First contact

          • Christian Schildwaechter

            The Oculus/Meta tutorials show an interesting progression. "First Contact" was the first one, starting on Rift CV1, with a large focus on "wow, I'm in VR" and (stationary) exploration, with the main interaction being putting disks into the replicator and watching the results. Very impressive, but not explaining a lot.

            They then went for the much more streamlined "First Steps", split into a very clear step-by-step tutorial part explaining the controllers, and two quasi-separate experiences, a small wave shooter and a dancing-with-robots experience to apply some of the new knowledge. Later "First Steps with Handtracking" provided exactly the same, only with hand tracking. As "First Steps" released on Quest 1, it had a visually much poorer environment than the packed room in "First Contact" due to the much slower hardware, but incorporated many of its interaction elements like the pull-string rockets.

            "First Hand" as the latest iteration, released alongside the Quest 3 but also running on Quest 2, combines the previous attempts and expands beyond them. There is again a more clear defined tutorial section, but everything is embedded in a much larger world with an actual story about restoring generators for a community of robots, with separate stages explaining aspects like controllers, locomotion, speech and hand tracking. It requires actually applying what is learned, with multiple locations to explore and simple puzzles to solve, making it much more of a game than its predecessors. It also takes 2-3x the time to finish.

            Unfortunately we don't know how well received these were. I'm pretty sure that almost 100% of the Rift CV1 users tried first contact, while only a small fraction of Quest 3/3S users will have run through "First Hands". Mostly because in 2016/17 VR was still new and unfamiliar, while most people who these days buy a Quest will already have tried one somewhere else, being already familiar with the basic interaction, plus a lot more media exposure for VR during the past decade.

            "First Contact" was probably the best in getting people interested quickly, "First Steps" the quickest introduction to basic mechanics without distractions, and "First Hands" the one that actually created the most practical experience with the medium. I'd really love to see Meta's internal data on how many users played through each, and statistics on how it impacted their later interaction with the platform.

    • Massimiliano Di Luca

      Hello Christian. Thank you for highlighting the Twiddler and in general chorded keyboard. We can definitively add it to the database. We have scraped discussion board and academic literature knowing and accepting that it is a fallible method. For this reason we created a database that can be updated every time we find devices that we didn’t know about and to future-proof the outcome of the research