We mentioned a few months ago how convincing the AI de-aging of Tom Hanks looked in Robert Zemeckis' new movie Here. Now we've learned more about how Miramax created the effect.
Visual effects supervisor Kevin Baillie turned to the creative studio Metaphysic, which digitally augmented performances using its own generative artificial intelligence technology and team of digital character artists. And the process even allowed actors to rehearse while watching their young selves perform. The results could replace the Irishman in our pick of the best CGI movie moments.
Written by Zemeckis and Eric Roth and based on an acclaimed graphic novel by Richard McGuire, Here tells the story of multiple families from one particular location. The story travels through generations, which meant having to digitally alter the apparent age of actors Hanks, Robin Wright, Paul Bettany and Kelly Reilly.
Baillie invited Metaphysic to screen test for the project back in 2022. For that, they recreated a young Tom Hanks, reminiscent of his appearance in Big while maintaining the emotional integrity of a contemporary performance by the actor.
Metaphysic’s proprietary process involves training a neural network model on a reference input, in this case video footage and images of a younger Hanks. A human artist then refines the results until the model is ready for production. From there, an actor or performer can drive the model, be it for a film or a live event.
As Here moved into production, Metaphysic trained the initial neural network models to generate photoreal younger versions of the actors. Metaphysic VFX supervisor Jo Plaete and his team ran the workflow in real-time during the shoot, providing a visual reference of what the performers would look like.
This allowed Zemeckis to provide direction while viewing both the raw camera feed and the digitally augmented feed with the actors’ younger faces with only about a six-frame delay. Metaphysic also set up a camera and monitor system that allowed the actors to rehearse while seeing their younger selves. This 'youth mirror' system had just a two-frame delay and provided the actors with feedback that helped them to fine-tune their performances to better match their younger selves.
Multiple models were trained for each actor to meet the needs of the film – Hanks is portrayed at five different ages, Wright at four ages, and Bettany and Riley at two each. Metaphysic says achieving this through traditional computer graphics techniques, involving 3D modeling, rendering, and facial capture, would have been impossible given the scale and quality required.
“Working with forward-thinking filmmakers like Bob Zemeckis and Kevin Baillie was a privilege. Their faith in this novel technology pushed us to new heights and allowed us to deliver on their ambitious vision,” Jo said.
While the neural network models used for the real-time outputs generated photoreal results, Metaphysic artists needed to enhance them to meet the standards of 4K footage for cinema. Metaphysic’s artists were also on hand to correct eyelines or tweak the emotions portrayed by each character.
Here was released by TriStar Pictures on November 1.
For more inspiration, see pick of the best upcoming VFX films of 2024.
