Ignore for a second how goofy the passthrough eyes on the front of Apple’s Vision Pro looks. I just spent 30 minutes with Apple’s $3,500 “spatial computer” (they’re not calling it a VR or AR or mixed reality headset) and holy f*cking god it is next level.
I’m not being hyperbolic. The Vision Pro is one of those I can’t believe this is real products that takes about five seconds to convince you that it’s magical. The mouse made the Mac, the click wheel made the iPod, the capacitive touchscreen made the iPhone and iPad, and for the Apple Vision Pro, eye and hand-tracking are the secret sauce.
This is not the janky and stuttery eye and hand-tracking found on the Oculus Quest Pro. It’s extremely accurate and responsive inputs that require little to no thought to navigate and control apps and experiences within the Vision Pro’s visionOS. Combined with the high-resolution visuals — virtual and color passthrough from the two 4K displays — the Vision Pro makes a strong impression that augmented reality with convincing depth really is the next step up in computing.
Vision Pro Hardware
As always, photos and videos don’t do the Vision Pro justice. It’s more compact and heavier than it appears online; Apple hasn’t shared official specs or dimensions yet so anything can change between now and early 2024 when it’s launches. Even if you’ve seen it from every angle in pictures, the materials come to life in person. The “singular piece of three-dimensionally formed laminated glass” is a sight to see and feel. Its reflective, fragile, and dense. All of the computer parts — the cameras and sensors, M2 and R1 chips, outward and inward displays, cooling system, etc. — are housed underneath its shiny finish. It’s not something you’re going to handle without care; you’ll feel its expensive custom components and materials every time you pick it up. It’s like a $6,000 Leica Q3, not a $600 Sony camera. It’s a M2 MacBook Pro, not a Chromebook. You get the idea.
The head band is the sleekest one I’ve ever seen on a headset. The 3D-knitted material tightens and loosens with the turn of the “Fit Dial” on the right. In my demo, I tried the headset with an additional top band. I have no idea how I looked and I didn’t care because the $3,500 device strapped to my face felt good and secure for the most part. I had to push the Vision Pro up a few times to keep it from sliding off my face, but only because I had moved my head around more than usual. A different light seal that fit more snug might have provided a tighter seal. Good thing they’re modular so users will have options.
There’s a button on the upper left for capturing spatial photos and videos with the built-in cameras; I didn’t get to use this in my demo. Across from it is a Digital Crown, lifted from the Apple Watch and AirPods Max, which is used to control the “Environment” or the opacity of your virtual space.
On each side of the temples, there’s an “audio pod” with two amplified drivers for Spatial Audio. They sound incredible. And then along the left side is a connector to the external battery connected with a braided cable. The two-hour-runtime battery either sits next to you on a sofa or in your pocket. I sat for almost the entirely of my 30-minute demo with the battery pack next to me on the sofa and other than one time when I forgot about the tether and the pack fell on the floor, the cable largely stayed out of my way.
Since I was wearing the Vision Pro, there was no way to see my eyes (with “EyeSight”) on the outer display. Overall, I’m impressed with the hardware. I dig the orange accents inside of the head band. I didn’t sweat at all with it on. That could, of course, be because Apple’s demo room was well air-conditioned and the fact that I wasn’t aggressively moving around like I would in a round of Beat Saber on a Quest headset. It’s no easy feat, but Apple really nailed the comfort with the Vision Pro. As something you wear on your face, comfort is incredibly important.
Almost As Sharp as Reality
Just as important as comfort is clarity. Using 12 cameras and five sensors — powered by the R1 chip — the Vision Pro creates a “passthrough” view of the real world that almost looks as a good as reality. I say almost because, even though each Micro-OLED display has a 4K resolution and the pixel density is high enough that you don’t see a “screen-door effect,” if you really look in the right places — like around the edges of someone’s hair or clothes — you can see the lightest of color fringing. Even so, it didn’t hinder the augmented reality experience. In fact, it’s the most advanced passthrough I’ve seen on any consumer headset; the Quest Pro’s color passthrough doesn’t even come close to the Vision Pro. An apt analogy would be: the Vision Pro visuals are a PlayStation 5 and the Quest 2/Quest Pro has jaggy PS1 graphics. It’s really that kind of difference.
Whether “fully immersed” (you can’t see the real world) or not, the displays render the virtual incredibly sharp. I completely forgot I was interacting with virtual windows most of the time. That’s how real the virtual objects looked. In other headsets, handling virtual objects on a 3D plane often is imperfect. A virtual object might flicker or fail to stay pinned in place, reminding you it’s not indeed tangible. Not so on the Vision Pro. Every virtual window or object that I saw within visionOS was detailed — text is actually crisp enough to read — and never glitched out, maintaining my suspension of disbelief. It’s this convincing visual fidelity that completely sucks you in. I couldn’t help but be amazed, and I suspect once people get try the Vision Pro for themselves, they’ll feel the same way.
Eye and Hand-Tracking in visionOS
I said earlier that it took all but less than five seconds for me to learn how to control visionOS and I meant it. Setting up the eye-tracking requires you to look at a series of points and hand-tracking just me holding my hands in front of the Vision Pro.
I was told to press the Digital Crown to bring me to the home screen — a familiar honeycomb of round app icons. To launch an app, I just looked at its icon and pinched it with my thumb and pointer fingers, and it floated before me. To drag app windows, pinch and drag where you want to move it. To scroll vertically or horizontally, just pinch and drag in the direction; put more speed into things and you get inertial scrolling just like on iPhone, iPad, and a TrackPad or Magic Mouse.
More than anything, I was most impressed by how accurate and precise the eye and hand-tracking was. For example, I thought I’d miss more when air tapping small buttons, but nope, I think I only missed once and that was probably my own error. Same for the eye-tracking; it’s super sensitive but somehow very accurate. Also, not once did the Vision Pro lose track of my hands, something that always happens on Quest headsets.
visionOS is so natural and intuitive to pick up that it almost feels too easy. I kept wanting to open more apps and flick them around. You would think a $3,500 headset should have more advanced inputs, right? Wrong. Great computer inputs are easy to understand and don’t require lengthy learning curves. That’s what Steve Jobs understood about the mouse, click wheel, and touchscreen. And Apple all these years later still understands this with eye and hand-tracking. This is clichéd, but Apple has basically made the Minority Report computer interface, where you control virtual interfaces with just your fingers. It’s nothing short of profound once you try it out. I couldn’t stop saying “wow” and “cool” and “This is sick” every single time I pinched a window to move it or pinched a button to “click” it.
Apple showed me one of many demos that only scratched the surface of what the Vision Pro is capable of. I watched a 3D video clip from Avatar: The Way of Water on a 100-foot-wide screen, sat in the best seat (dead center) inside of the virtual theater, as the Na’vi rode skimwings, convincingly popping off the screen just like in a 3D movie theater. Could this be the return of 3D movies?
A virtual dinosaur walked out of a wall and turned its head in the direction I looked at. A 3D butterfly flew over and landed on my hand. I did a one-minute meditation session where an orb of petals exploded peacefully before me and then reassembled itself, all while blocking out the real world by spinning the Digital Crown to adjust the Environment to my preferred opacity.
I had a FaceTime call with another person wearing a Vision Pro and saw his “Persona” — or as Apple calls it “a digital representation of themselves created using Apple’s most advanced machine learning techniques, which reflects face and hand movements in real time” — look directly in my eyes. I found the uncanny valley recreation somewhat unsettling; I whipped the 2D window he was in in all directions and listened as the sound of his voice accurately moved accordingly; I almost fell over leaning too far to get a closer view of a 3D model of a room he was showing me in the Freeform app.
I flipped through several photo and video types. Two “spatial photos” where it felt like a moment in time, with real depth, was preserved. In one photo a child blew out candles on a cake and I reached out only to remember that it was virtual; in another spatial photo, I could almost feel the crackling of a campfire. I love Live Photos so it was only natural that I enjoyed these spatial memories. I got to see panoramas shot on iPhone expanded to wrap around me in detailed resolution. Apple’s new “Immersive Video” format is a 3D video recording with a 180-degree field of view, and man, some of these were captured from such interesting angles like the dugout of a baseball field or the center of a tightrope across a mountain, that I expected to get nauseous or experience some kind of motion sickness. But I never did. I’m not saying there won’t be people who don’t get nauseous, but I didn’t and I normally get a little queasy just panning my head at these kinds of photos in VR. It’s a real testament to all the ways Apple is combatting perceptual latency.
None of these demos were physical in the way they are in virtual reality with headsets like the Quest 2 or Quest Pro, and that’s by design. Because the Vision Pro is a computer and you sit or stand in front of “real” computers. Sure, Apple is touting playing Apple Arcade games on a virtual display with an Xbox or PlayStation controller, but note the controller. You sit down and play these gamepad-controlled titles. It’s possible we’ll see more advanced types of games akin to what you get on VR headsets, but for now, I’m fine if Apple leaves that to the Quest 3 or PlayStation VR 2.
My Mind Is Blown
I’ve tried almost every major virtual reality or mixed reality headset there is. None come remotely close to being as polished as the Vision Pro. Not on hardware and definitely not on software. Apple has a huge advantage with visionOS and the massive developer community from its iOS and macOS platforms. Hundreds of thousands of iPad and iPhone apps will just work in 2D inside of the Vision Pro, which automatically makes it a more familiar computer than any VR headset. And if developers choose to tweak them for spatial computing, they’ll work even better.
There’s still so much more to the Vision Pro that Apple hasn’t made available for anyone to try. But even with the controlled and limited demos, I’m very optimistic that Apple is onto something here. The hardware will no doubt get better and hopefully cheaper generation over generation. But the platform Apple’s making with visionOS and the eye and hand-tracking (and voice which I didn’t get to try) is what’s really going to make or break the device.
I see tons of potential, and the only hurdles are the $3,500 price tag and two-hour battery life. I know a lot of people are horrified at the price, which is a lot, but it also feels kind of low for something so state of the art. I know that sounds crazy, but when you consider how the Vision Pro goes beyond 2D computing and you add up the equivalent of multiple physical displays, 3D cameras, a theater-sized screen, surround sound speakers, and hand and eye-tracking, it’s not as absurd as it sounds.
