Apple Vision Pro may not ship with controllers, but a newly published patent application shows that controllers are still being actively investigated by the company.

Spotted by Patently Apple, an Apple patent application published today reveals the company is still actively investigating controllers for XR input, including stylus-like devices with dial inputs.

The new patent application, titled Computer Systems with Handheld Controllers describes “one or more handheld controllers to interact with a virtual reality or mixed reality system (e.g., a head-mounted device with a display), to supply input to a desktop computer, tablet computer, cellular telephone, watch, ear buds, or other accessory, to control household items such as lighting, televisions, thermostats, appliances, etc., or to interact with other electronic equipment.”

Photos in the patent show several concepts for such a controller, including stylus-like devices and wand-like devices. The patent application says the controllers could use various sensors for detecting user intent, including touch-sensitive surfaces, buttons, or dials.

 

There’s even a concept for a wand-like device that splits into two controllers for dual-hand input.

“The housing may have a first and second housing portions operable in a first mode in which the first and second housing portions are mated with one another and a second mode in which the first and second housing portions are separated from one another to form independent hand-held controller devices,” the application reads.

Though it was published today the patent application (identified as US20240012496A1) was first filed by Apple in mid-2023. Of course, patent applications don’t prove a company is actively working on an actual product. It’s common practice for large companies to secure future IP ‘just in case’ they ever choose to pursue it or to box out competitors.

SEE ALSO
SteamVR Driver for Vision Pro Now in Development, Including VR Controller Support

However, we have a strong feeling that a stylus-like controller isn’t off the table for Vision Pro in the future. Apple will surely want to enable professional design workflows which almost universally require a precise pointing device—the same reason they eventually released the Apple Pencil stylus for iPads.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • Andrey

    EXCUSE ME!
    It’s not a “controller”, but a “Spatial Computing Advanced Manipulator” (c) – or “SCAM” (c) for short. How dare you call it a mere controller?!
    And how dare all those totally not innovative people at the patent bureau mention VR and AR in the patent regarding this SCAM?! Don’t they understand that “Spatial Computing” (c) is TOTALLY different than VR, AR, XR and MR?!
    I truly hope Apple will sue all of them for such blasphemy!

    • Nevets

      Unlike many readers here, Apple are shrewd operators. Most, if not all, of their judgement calls have resulted in successful new product categories.

      • ViRGiN

        Power Mac G4 Cube was an absolute flop, but then again, 2000 Apple isn’t the 2024 Apple.

  • I think Apple almost for sure will release a sort of pencil, considering its target market of designers

    • Arno van Wingerde

      But then you need something to write on, say a projected piece of paper on your real table. Writing on a projected piece of paper in space sounds difficult, without a feeling for where the paper is: you would simply push the pen through the virtual paper. I have not tried apps like Vermillion on the Quest3, but that would have the same problem.

      • J Dolphin

        Vermillion works surprisingly well without an actual physical surface.. But I suspect this is because it specifically emulates Oil painting… a painter typically “pats” the canvas with the brush, as opposed to a stylus/pen, where much more force is exerted to draw or write.

  • Nevets

    Of course they’re not off the table. Apple is just doing its usual thing of pushing out a nice but artificially constrained product, and then adding the commonplace features everybody needs as exciting upgrades. The original iPhone didn’t have copy and paste. Although I acknowledge the reasons for the AVP gimping are more nuanced and connected with their intended introduction and positioning of the use cases they think will drive initial adoption.

  • Christian Schildwaechter

    The majority of apps that AVP owners will use will be iPad apps, and it will take some time for XR apps to appear. Apple launched the Apple Pencil as an iPad accessory in 2015, and it is used by a lot of drawing apps, so a similar accessory for AVP could enable using similar apps where fingers don’t provide sufficiently fine input.

    Not sure how exactly that would work, if AVP would project an image onto a flat surface to draw on, or if you’d use it like a graphics tablet. But with pen and keyboard being the extra iPad input devices Apple offers, offering the same for the AVP running iPad apps makes sense.

  • Ad

    VR has always needed a really good pen tool for some amount of dexterous control for creative software and writing.