Even having tried the Apple Vision Pro briefly in three separate sessions, we still have yet to see how certain parts of the visionOS software works. You can read my hands-on with the Vision Pro at WWDC and my experience viewing spatial video and other media content here and here, but if you’re curious about what air typing on the virtual keyboard is like, just watch YouTuber Brian Tong’s video below.
Tong’s thoughts on air typing start at around 14:05 into the video. He says there are two ways to interact with the keyboard.
“I can either look at the specific letters on the keys and then pinch my finger for each letter or I can actually kind of bring it up with that window bar and then peck away in the air,” Tong says.
He also says there’s a “haptic sound” when a virtual key is pressed, which could help with accuracy, though a sound is unlikely to be a replacement for a tactile key press. Good thing you can connect a wireless keyboard to the Apple Vision Pro or use dictation!
Although Tong says he wasn’t able to type very quickly using the virtual keyboard, he cautioned that visionOS was still in development, so things could be improved before Vision Pro comes out next year. On his overall experience typing on the keyboard, Tong concluded: “It was good; it was decent; it wasn't bad. I was able to do it.”
For a deeper look at how typing works, definitely check out the “Design for Spatial Input” video that Apple released at WWDC.
“Every time we touch something in the physical world, our hands receive lots of multisensory feedback, which is essential to our perception,” explains Apple designer Eugene Krivoruchko. “None of this is happening when we reach out and touch virtual content. And to make that interaction work, we need to compensate for the missing sensory information with other types of feedback.”
Krivoruchko says the virtual keyboard buttons are “raised above the platter to invite pushing them directly” and that there are “hover” and “highlight” states that work as proximity cues to guide your fingers. Keyboard buttons get brighter as your fingers approach them.
Apple seems to have put a lot of thought into the virtual keyboard, but we’ll have to see how well it works for ourselves. Remember how Steve Jobs made it seem like typing on an iPad’s glass screen with two hands, like you would on a laptop or desktop keyboard, couldn’t be simpler? Everyone hated it.
