Would reality really be complete without our beloved four-legged friends? Certainly not. Luckily the latest update to Apple’s ‘Vision’ framework—which gives developers a bunch of useful computer vision tools for iOS and iPad apps—includes the ability to identify and track the skeletal position of dogs and cats.

At Apple’s annual WWDC the company posted a session introducing developers to the new animal tracking capabilities in the Vision developer tool, and explained that the system can work on videos in real-time and on photos.

The system, which is also capable of tracking the skeletal position of people, gives developers six tracked ‘joint groups’ to work with, which collectively describe the position of the animal’s body.

Image courtesy Apple

Tracked joint groups include:

  • Head: Ears, Eyes, Nose
  • Front Legs: Right leg, Left leg
  • Hind Legs: Right rear leg, Left rear leg
  • Tail: Tail start, Tail middle, Tail end
  • Trunk (neck)
  • All (contains all tracked points representing a complete skeletal pose)

Yes, you read that right, the system has ‘tail tracking’ and ‘ear tracking’ so your dog’s tail wags and floppy ears won’t be missed.

The system supports up to two animals in the scene at one time and, in additional to tracking their position, can also identify a cat from a dog… just in case you have trouble with that.

Image courtesy Apple

Despite the similarity in name to the Vision Pro headset, it isn’t yet clear if Apple will expose the ‘Vision’ computer vision framework to developers of the headset, but it may well be the same foundation that allows the device to identify people in the room around you and fade them into the virtual view so you can talk to them.

SEE ALSO
Arcade Management Sim 'Arcade Paradise VR' Coming to Quest Later This Month

That may have also been a reason for building out this animal tracking system in the first place—so you don’t trip over fido when you’re dancing around the room in your new Vision Pro headset—though we haven’t been able to confirm that system will work with pets just yet.

Newsletter graphic

This article may contain affiliate links. If you click an affiliate link and buy a product we may receive a small commission which helps support the publication. More information.


Ben is the world's most senior professional analyst solely dedicated to the XR industry, having founded Road to VR in 2011—a year before the Oculus Kickstarter sparked a resurgence that led to the modern XR landscape. He has authored more than 3,000 articles chronicling the evolution of the XR industry over more than a decade. With that unique perspective, Ben has been consistently recognized as one of the most influential voices in XR, giving keynotes and joining panel and podcast discussions at key industry events. He is a self-described "journalist and analyst, not evangelist."
  • ViRGiN

    full body tracking is near.
    none of that pcvr-elitists bullshit disguised as dog trackers aka HTC trackers.

  • mate

    You will be able to equip your pet with armour and helmet in AR, how cool is that xD

    • Sven Viking

      * Pet armour DLC sold separately.

      • Kenny Thompson

        *Via the App Store monopoly.

  • Sven Viking

    HTC’s promised pet tracking, finally realised. :)

  • xyzs

    Damn, my pet alligator won’t be recognized ?

  • david vincent

    And what about the other animals… Like children ?

  • Oh wow, this is cool!