Apple Vision Pro is a mixed-reality headset—which the company hopes is a "revolutionary spatial computer that transforms how people work, collaborate, connect, relive memories, and enjoy entertainment"—that begins shipping to the public (in the United States) later this week.

Critics have doubted the appeal of the face-worn computer, which "seamlessly blends digital content with the physical world," but Apple has pre-sold as many as 180,000 of the US$3,500 gizmos.

What does Apple think people will do with these pricey peripherals? While uses will evolve, Apple is focusing attention on watching TV and movies, editing and reliving "memories," and—perhaps most importantly for the product's success—having its customers not look like total weirdos.

Apple hopes the new device will redefine personal computing, like the iPhone did 16 years ago, and Macintosh did 40 years ago. But if it succeeds, it will also redefine concerns about privacy, as it captures enormous amounts of data about users and their environments, creating an unprecedented kind of "biospatial surveillance".

Spatial computing

Apple is careful about its brand and how it packages and describes its products. In an extensive set of rules for developers, the company insists the new headset is not to be referred to as a "headset." What's more, the Apple Vision Pro does not do "augmented reality (AR), virtual reality (VR), extended reality (XR), or mixed reality (MR)"—it is a gateway to "spatial computing."

Spatial computing, as sketched out in the 2003 Ph.D. thesis of US software engineer Simon Greenwold, is: "human interaction with a machine in which the machine retains and manipulates referents to real objects and spaces." In other words, the computer can interact with things in the user's physical surroundings in real time to provide new types of experiences.

The Vision Pro has big shoes to fill for new user experiences. The iPhone's initial "killer apps" were clear: the internet in your pocket (including portable access to Google Maps), all your music on a touch screen, and "visual voicemail".

Sixteen years later, all three of these seem unremarkable. Apple has sold billions of iPhones, and some 80% of humans now use a smartphone. Their success has all but killed off earlier tools like paper maps and music CDs (and the ubiquity of text, image and video messaging has largely done away with voicemail itself).

Killer apps

We don't yet know what the killer apps of spatial computing might be—if any—but here is where Apple is pointing our attention.

The first is entertainment: the Vision Pro promises "the ultimate personal theater".

The second is an attempt to solve the social problem of walking around with a weird headset covering half your face. An external screen on the goggles shows a constantly updated representation of your eyes to offer important social cues about your gaze to those around you. Admittedly, this looks weird. But Apple hopes it is less weird and more useful than trying to interact with humans wearing blank aluminum ski goggles.

The third is the ability to capture and and relive "memories": recording and playback of 3D visual and audio from real events. Reviewers have found it striking: "this was stuff from my own life, my own memories. I was playing back experiences I had already lived."

Apple has patented tools to select, store, and annotate digital "memories." These memories are files, and potentially products, to be shared in "spatial videos" recorded on the latest iPhones.

This article is republished from The Conversation under a Creative Commons license. Read the original article.