Disney researchers have created a new neural network that can alter the visual age of actors in TV or film, reports Gizmodo. The technology will allow TV or film producers to make actors appear older or younger using an automated process that will be less costly and time-consuming than previous methods.
Traditionally, when special effects staff on a video or film production need to make an actor look older or younger (a technique Disney calls “re-aging”), they typically either use a 3D scanning and 3D modeling process or a 2D frame-by -frame digital retouching of the actor’s face using tools similar to Photoshop. This process can take weeks or longer, depending on the length of the work.
By contrast, Disney’s new AI technique, called Face Re-aging Network (FRAN), automates the process. Disney calls it “the first practical, fully automatic, and production-ready method for re-aging faces in video images.”
A research demonstration video for Disney’s re-aging technology.
To build FRAN, Disney researchers randomly generated thousands of examples of synthetically aged faces between ages 18 and 85 using StyleGAN2. With that training data in hand, FRAN learned general principles about how a person’s appearance changes with age. Now that the training is complete, it can apply those aging principles to a real actor in motion, frame by frame.
“Our network is trained in a supervised fashion on a large number of face image pairs showing the same synthetic and photorealistic person, labeled with the corresponding source and target ages,” the researchers wrote in a corresponding academic paper. By generating the training data synthetically, they bypassed the “seemingly impossible task” of collecting images depicting “a variety of identities, ages, and ethnicities in different viewpoints.”
The result is what Disney calls a “production ready” solution—which means it creates high-enough quality output to be used in a real film or TV show. It’s possibly the first AI solution of its kind that can dynamically alter an actor’s age on video despite variable expressions, lighting conditions, and viewpoints. The researchers also developed a user-friendly interface for FRAN that will allow artists to easily use the tool in a production environment.
Disney presented the research in a paper titled “Production-Ready Face Re-Aging for Visual Effects” on Wednesday and submitted it for inclusion at the 2022 ACM SIGGRAPH Asia conference in December. The paper’s authors include Gaspard Zoss, Prashanth Chandran, Eftychios Sifakis, Markus Gross, Paulo Gotardo, and Derek Bradley, all affiliated with Disney Research Studios in Zurich, Switzerland.
Considering Disney’s history with inserting computer-generated actors into Star Wars films and TV shows, including some that have been de-aged using CGI, we would not be surprised to see technology similar to FRAN widely used in future Disney productions, although no plans have been announced.