After years of speculation, Apple is getting into VR and AR in a big way. At its WWDC 2023 developer conference keynote, the company revealed Vision Pro, an XR headset combining virtual and augmented reality. It's Apple's first big hardware category since the Apple Watch was released in 2015. It will start at $3,499 next year and will ship on Apple.com and in US retail stores. It's being shown now to get developers involved.
The headset has a single piece of laminated glass with a "wide array of cameras and sensors." The frame includes a button for photos and videos, and a digital camera to switch between levels of AR and VR. Vision Pro's frame has a thermal design that draws air through holes in the system to keep it cool. Needless to say, this isn't a fanless design, although Apple says that it's "near silent" in operation.
Apple is calling it "the most advanced personal electronics device over" and says it is based on over 5,000 patents.
The light seal comes in multiple sizes and conforms to your face. Straps reach out from the frame with "Audio Pods" to deliver spatial audio. The headband is a ribbed, 3D pressure that stretches with an attachable mechanism. An adjustment dial lets you make small adjustments during use.
Glasses-wearers will use inserts made by Zeiss that attach to the lenses. There was no mention of the cost for these accessories for an already pricey headset.
Apple promises two hours of battery on a charge (but it can be used longer while plugged in). The hardware includes a battery pack wired to the headset, which you then put in a belt or pocket.
An external display will show your eyes so that people can recognize if you're immersed in VR or not, and keep people from feeling isolated.
We got to see the design form a distance at WWDC. It's space-age, and the reflective glass along the front is surprisingly welcoming. It's a very attractive device, but wearing it is still a bit of a mystery. Apple ensured these models weren't touched, but we did get a better look at the battery that hangs off the side of the device.
Tech in Apple Vision Pro
Vision Pro will boast some impressive specs. It fits 64 pixels in the space of a single iPhone pixel. It has 23 million pixels across two panels for more than 4K resolution. It supports high dynamic range and allows for sharp text, Apple states.
Apple uses a dual-chip design with Vision Pro, with M2 and a new chip called R1. R1 is designed for input from 12 cameras, five sensors and six microphones. M2 allows for performance, while R1 is designed to make experiences feel immersive.
Meta's $1,499 Quest Pro has two LCD panels at 1800 x 1920 pixels with a claimed 106-degree horizontal field of view and a 96-degree vertical field of view. (Last week, Meta announced the Meta Quest 3, but didn't provide full specs ahead of a full launch later this year). And yes, there's a screen on the outside to show your eyes.
The Audio Pods trace your room with sound to customize it for you. Apple calls this "audio ray tracing." There are tons of scanners, including LiDAR, for 3D tracking. There are IR cameras and LEDs to track your eyes, so you don't need "clumsy hardware controllers."
For FaceTime, Apple creates a digital version of your face. After being scanned by the front cameras, Apple uses a neural network to make a 3D avatar to use in FaceTime calls. When someone in another Vision Pro is chatting with you, they'll see it in 3D.
The new OS is called visionOS, which supports iOS framework, a new real-time subsystem, a foveated rendering pipeline and a multi-app 3D engine to keep multiple apps running in the same simulation.
For security, Apple is using Optic ID, a new take on features like Face ID and Touch ID. Your iris scan will be stored on-device in the secure enclave. Apple says eye input will be separated into a special background process so that websites or apps won't be able to see where you're looking. Only finger taps will be registered.
Using Apple Vision Pro
Apple's operating system for AR and VR will be somewhat familiar to users of iOS and iPadOS with a home screen with app icons and widgets that can be rearranged. There's a home view in your space, with apps that appear as they float in your room. They respond to light and cast shadows on your home. You can scale apps and place them wherever in your space that you wish.
The switch between AR and VR is critical here. Apple relies on a mix of the two so that people who use the headset don't feel entirely blocked out from the world. The dial, similar to the digital crown on the Apple Watch, lets you move in between AR and VR modes.
The headset relies solely on your eyes, hands, and voice. There's no controller. Graphical elements on the screen respond to your looking at them, and a tap of the fingers selects. You can flick your hands to scroll.
Apple uses the EyeSight moniker to show your eyes to others near you on an external screen. The company says this keeps you from being isolated from others and lets others know if you're immersed in VR or can see in AR.
For productivity, you get a "canvas for apps." You can place your apps anywhere on the screen. It looks like a floating version of Mac, with web browsing and other productivity apps around in space. You can turn your head to move from app to app or even put 3D objects in front of you and see them from any angle.
Vision Pro works with the Magic Trackpad, Magic Keyboard and other Bluetooth accessories. You can also use your Mac to put your Mac's screen on the headset for a massive 4K display.
There's also a version of FaceTime that lets you see digital versions of other people in one-on-one calls. When new people are added, they float in front of you. You can share apps with others while on a call.
Multimedia and Gaming on Vision Pro
Apple showed it as a multimedia machine for use when you're not working. Photos float in mid-air, and panoramas taken from iPhone will wrap around you. You can take pictures or video with the press of a button on the unit to watch in 3D later. Vision Pro blinks to let others see you're taking images.
Vision Pro also works as a personal movie theater. You can adjust the size in your space, and the OS automatically dims the surrounding area while adding spatial audio. You can even add an "environment" to work as a massive movie theater larger than your home. A specialized movie theater is meant to focus on film and nothing else. You can also watch 3D "spatial" movies; Apple showed off Avatar: The Way of the Water.
You can also use the Vision Pro to play Apple Arcade games with game controller support. Apple promises hundreds of titles on launch day.
Disney CEO Bob Iger joined Apple CEO Tim Cook at WWDC, showing off Disney Plus, including customized versions of shows like The Mandalorian, as well as customized ESPN game views that show replays in 3D. If also showed a Marvel: What If? game that seems to put you in the shoes of a superhero. Apple clearly has partners ready for this headset — Disney Plus will be ready on a launch day, Iger said.
Developing for Apple Vision Pro
Some developers have already had their hands on Vision Pro. For example, Apple demoed a 3D beating heart as well as a design of an Alfa Romeo F1 car. These are similar to what we've seen on other headsets and even Microsoft's HoloLens.
But others sound more unique, like a spatial interface for DJ-ing and a sky guide to transform Vision Pro into a planetarium. And Microsoft has its Office apps ready to go, as do a bunch of video conferencing software companies.
The same frameworks in iOS and iPadOS are built into visionOS, meaning plenty of existing apps should largely work out of the box with the built-in controls.
Apple has been working with Unity, and it says games and other apps based on Unity will be able to work with minor tweaks. There will be a brand new App Store to see everything built for Vision Pro as well as any other compatible apps.
