The Apple Vision Pro reveal was noteworthy for a number of reasons, not the least of which was the revelation that the AR device does not have physical controllers. In a developer session posted by Apple, Vision Pro developers give the inside scoop on how users will interact with and control the upcoming headset.
Apple begins the session by emphasizing what was mentioned during the reveal, which was that the company envisions the majority of menu control with the Vision Pro being driven by where users direct their line of vision in combination with small hand gestures. That said, the developers concede that “some tasks are better suited to interact directly,” which means Apple’s “direct touch” feature where users reach out and directly interact with UI elements, an option for a connected physical keyboard, or the Vision Pro’s virtual keyboard.
In speaking about the virtual keyboard, designer Eugene Krivoruchko mentions that, to make up for the tactile response of a physical keyboard, additional cues are needed for the virtual keyboard to feel intuitive. “While the finger is above the keyboard, buttons display a hover state and a highlight that gets brighter as you approach the button surface,” Krivoruchko explains. “It provides a proximity cue and helps guide the finger to target. At the moment of contact, the state change is quick and responsive, and is accompanied by matching spatial sound effect.”
And, of course, there is voice control. The developers explain that, to prompt a voice command, users simply need to focus their eyes on a microphone icon in the Vision Pro’s UI to engage a “speak to search” feature. This opens up the device’s numerous microphones for voice input.
