AI in Dubbing: Almost There, But Still a Long Way to Go!(Artificial Intelligence Talks)

Author:Barreau, Jacques

AI has already been implemented into our lives. It's in our cars, our telephones, our computers, our transportation systems, and our hospitals. AI is helping us make decisions daily. But what about language-dubbing?

Although a new AI-based system is trying to change the facial expressions of actors to accurately match dubbed voices, AI can also help the dubbing process by helping the dubbing actors themselves--without altering the onscreen content--which could lead to a more complicated distribution system for a TV product.

Translation Memory is the first AI stage in the dubbing process and very different from Automatic Computer Translation, also known as Phrase-Based Machine Translation, wherein the machine translates a phrase word for word, without understanding the context.

Unlike humans, machines will remember every word of a text, and consequently, are perfectly suited to build glossaries for each and every episode of a TV series. Since human mistakes will be avoided, character names and/or locations can then accurately be reused.

The next step: Neural Machine Translation (NMT), which will bring the predictive concept into play. This is a piece of software using different algorithms working together, like the neurons in our brain. The same way a child learns a language, the machine will learn to predict complex sentences, after first memorizing expressions, grammar, and linguistic rules. Furthermore, NMT is able to learn new languages by itself.

Obviously, translating a narration in a documentary is very different from translating dialogue in a TV series. And comedies will always be more complicated to translate as we need to adapt the original humor to the localized version.

We can say today that in some cases, depending on the complexity of the dialogue, the machine will get much better and the need to have a human editing will already be reduced significantly after the third episode of a TV series.

Today, the NMT concept is still in the infancy stage, as we are just now seeing machines capable of learning how humans think. But how many years will it take for machines to understand how humans think after they begin to translate from one language to another, after adapting the sense of humor for different cultures, and after figuring out the degree of emotion necessary for complex dialogues?

The shorter process time requested by the broadcasters so that they can air the subtitled or dubbed versions earlier and earlier are making AI a...

To continue reading