Apple's Vision Pro headset features an advanced version of Siri that can recognize and describe the user's surroundings, enhancing the mixed reality experience. While the device provides a virtual clock and panoramic views of Paris, these elements are overlaid digitally, contrasting with the actual physical bookshelves present in the environment. This new iteration of Siri is designed to respond seamlessly when users invoke it, even without the need for a camera app, simply by saying, "Hey, Siri."
Revealed during Apple's recent WWDC event, this early preview of VisionOS 27 demonstrates the potential of the headset's capabilities, though some aspects remain under development. The Vision Pro aims to expand Siri's visual intelligence across various Apple devices, including iPhones, iPads, and Watches, with a rollout expected this fall.
Siri's ability to interact with the environment through eye tracking and spatial graphics is a significant leap forward, allowing it to provide real-time information about the user's immediate surroundings. Although the technology is still in its early stages, its potential for future applications in augmented and mixed reality environments is considerable.