Fool the eye: A portrait of a bride-to-be reveals computational photography’s dark side

A woman in a wedding dress in front of a double mirror.

While Tessa Coates was trying on wedding dresses last month, she posted a seemingly straightforward snapshot of herself on Instagram before a double mirror. As those who have been through the wedding process know, such a mirror gives two views of a bride. Since the photograph was shot from behind Coates -- an actress and comedian in the U.K -- with her iPhone, you could see much of her and her dress from the back, giving you a third view.

The photo depicts what appears to be a bride simply viewing herself in a prospective wedding dress, but upon closer inspection, something looks off. In fact, when you study Coates’ arms and hands, you soon realize that they are all positioned differently from each other. In other words, neither mirror is reflecting the image of Coates that we see from the back in the photograph. According to the story Coates tells on Instagram, when she looked at the photo (which she said wasn’t Photoshopped), she had a full panic attack. She even went back to ask the dress shop owner if the mirrors were taking and displaying video. (They weren’t.)

She became more anxious and alarmed after showing and discussing it with friends that day. She then showed it to a number of people and employees in a nearby Apple store she visited. On Instagram, she joked that perhaps she really had magically crossed over and lived “in the mirror realm now,” or that she was “on the second layer of ‘Inception.’”

But had Coates in fact followed Alice through the looking glass?

What kind of photo did this bride-to-be have on her iPhone?   

If this were a typical iPhone snap shot, the bride's pose would be the same from all three views. But as you can see (I've added red arrows for clarity), the bride's arms and hands are positioned differently in each of the two mirror images and the view of her from the back. (Image credit: wheatpraylove Instagram)

At that Apple store, she finally was able to meet with an Apple genius, named Roger, who was able to provide her with answers. 

“First of all, an iPhone is not a camera, but a computer,” he explained. When it takes a photo, it captures a series or burst of images … even when it’s not explicitly taking a panorama, live photo, or burst of images. “It takes a series of images from right to left.” In this case, at the exact moment it crossed behind her back, Roger said, “You raised your arms, and [your iPhone] made a different image on the other side.” 

In that split-second moment, Roger said, the camera made an “AI decision and stitched those two photos together.”  He also offered that Apple was beta testing this new feature (presently found only on Google phones), Coates said. Lastly, Coates said Roger noted it was a million-in-one chance that it would stitch the photo right at the moment she raised her arms.

On Instagram and elsewhere, there have been lots of theories posted on how or why this happened, including some who are skeptical of the entire story. But one tech reviewer, named Faruk, who has a YouTube channel called iPhonedo, describes why he believes it’s not Photoshopped, and that it is, in fact, a panorama.

What is computational photography?

Although it's not clear exactly what type of photo was captured or what kind of mode Coates had enabled on her phone (an iPhone 12), what's clear is that it’s a computational photograph, since it was captured with an iPhone.

So what exactly is computational photography? As the name implies, computational photography uses computer processing power to harness data from an image sensor (or multiple image sensors) in various ways to enhance traditional photography techniques.

It's also used to develop new photography techniques and forms. For instance, with traditional film photography, photographers could produce panoramas, but it was very labor intensive and difficult to create panoramas, even ones that included just a few images. Because iPhones are not only digital, but are empowered with computer algorithms, they can quickly stitch scores of images together to create panoramics.

But iPhones and other smartphones use computational photography for lots more; they also use it to produce high-dynamic range (HDR) images, photos shot in portrait modes, and other innovative digital genres that combine computer power and photo optics. All of these are examples of computational photographs. 

These two panoramas, shot on my iPhone 12, are examples of computational photography. The top image depicts the U.S. Capitol in Washington, D.C., and the bottom shows the Parthenon on the Acropolis in Athens. In each image, I panned my iPhone as it captured scores or perhaps hundreds of images. It then digitally stitched the best parts of them together.  (Image credit: Future, photo by Terry Sullivan)

How to take good panoramas on your Apple iPhone

We can’t be sure exactly what type of beta mode Coates’ phone was in (if it was, in fact, in a beta or testing mode). Some comments on the internet suggest that the image might have been created in panorama mode, which lets you capture extremely wide photos of particular scenes, landscapes, or cityscapes.

On iPhones, you can enable this mode by opening the camera app and swiping left until you find “PANO.” Here are a few tips to get better results with your panoramas:

Check the direction you're panning: To create a horizontal panorama, hold your iPhone vertically and then pan either right-to-left or left-to-right. Make sure the arrow points in the direction you’re panning. To create a vertical panorama, you’ll hold your iPhone horizontally and pan upwards or downwards.

Avoid moving subjects: When shooting panoramas, it’s best not to capture subjects that move, since you’ll often end up with odd or weird aberrations or distorted subjects.

Practice, practice, practice: Before you create your final panorama, take some time to create a few test panoramas. For starters, it will allow you to figure out if you're panning too slowly or too quickly. It will also help you work out other compositional issues, such as if the lighting is correct, if there are subjects that are moving, etc. 

This image is actually part of a larger panorama, which can sometimes include unintended distortions and aberrations. Because the two central figures were in motion, the camera was unable to capture a sharp image of them. The iPhone's algorithms then unsuccessfully attempted to merge the figures with the others ... unless you like a touch of surrealism. (Image credit: Future, photo by Terry Sullivan)

Should we be worried about computational photography?  

There are some aspects of Tessa Coates’ story that sound comical—she is an actress and a comedian, after all—but other elements were alarming. This is partly due to the fact that although computational photography appears similar to traditional photography, in many ways it’s very different. And we don’t always know if we can recognize those differences.

Take the iPhone's portrait mode. An iPhone using this computational photography mode to capture a person’s face will most often accurately focus on the person’s head and only blur the background to replicate bokeh found in traditional film photography. But unlike a film camera, you can’t use the feature on still-life subjects. For example, when I tried to shoot an evergreen branch set against a background of similarly colored green trees, it severely cropped my photo, creating an artificial looking branch. Of course, portrait mode was never meant to capture an evergreen branch outside set again a background of green trees. But with an analog film SLR camera, I could apply the same rules of shallow depth of field to my outdoor evergreen photo that I could when shooting a portrait.

What’s important to understand is that by using computational photography, tech companies have dramatically altered some of the essential rules, principles, and techniques of photography. My concern is not that they're changing these rules, but that many of those changes will remain hidden from us … as when Coates' iPhone captured a disturbing composite photograph of her wearing a wedding dress.

Will your next photograph prove as startling?

This composite image has four photos showing the effects of the iPhone's portrait mode: On the left side, there are two portraits of my son (the bottom image was taken in portrait mode and the top image was not). Notice how the details in the background are blurred, as if it was taken with a high quality lens (with a wide aperture) on a mirrorless camera. In the top image, most of the background details are more or less sharp. On the right side, you'll see an evergreen branch with green trees in the background. In the bottom image, because the camera didn't recognize a face, the algorithms in the portrait mode didn't "know" how to separate the details in the background from those in the foreground. In effect, my iPhone's computer became confused. The result was an odd and drastic crop of sorts to the branch in the foreground. (Image credit: Future, photos by Terry Sullivan)