a company called Flawless has created an AI-powered solution that will replace an actor’s facial performance to match the words in a film dubbed for foreign audiences.
[…]
What Flawless is promising to do with its TrueSync software is use the same tools responsible for deepfake videos to manipulate and adjust an actor’s face in a film so that the movements of their mouths, and in turn the muscles in their faces, more closely match how they’d move were the original performance given in the language a foreign audience is hearing. So even though an actor shot a film in English, to a moviegoer in Berlin watching the film dubbed in German, it would appear as if all of the actors were actually speaking German.
[…]
Is it necessary? That’s certainly up for debate. The recent Academy Award-winning film Parasite resurfaced the debate over dubbing a foreign film versus simply watching it with subtitles. One side feels that an endless string of text over a film is distracting and takes the focus away from everything else happening on screen, while the other side feels that a dub performed by even a talented and seasoned voice artist simply can’t match or recreate the emotions behind the original actor’s performance, and hearing it, even if the words aren’t understood, is important to enjoying their performance as a whole.
[…]
The company has shared a few examples of what the TrueSync tool is capable of on its website, and sure enough, Tom Hanks appears to be speaking flawless Japanese in Forrest Gump.
[…]
Source: Flawless Is Using Deepfake Tech to Dub Foreign Films
Robin Edgar
Organisational Structures | Technology and Science | Military, IT and Lifestyle consultancy | Social, Broadcast & Cross Media | Flying aircraft