AI made me speak 5 languages in one click. Here's how you can do it too.
And what it means for the future of media
Welcome back to Year 2049, your source of practical insights, case studies, and resources to help you embrace and harness the power of AI in your life, work, and business.
If this was forwarded to you, you can subscribe for free to receive Year 2049 in your inbox every Friday.
Hola! Ciao! Olá! Hallo! Witam!
AI has officially turned me into Mr. Worldwide.
If you missed the video I posted on social media earlier this week, your mind is about to be blown:
Tiktok failed to load.Enable 3rd party cookies or use another browser
(Alternative links: TikTok | Instagram | YouTube)
I made this video after seeing the viral clip of Lionel Messi speaking English. I’ve been watching Messi since I was 8 years old and always had to rely on subtitles or translators speaking over him to understand what he was saying. Hearing him speak English in his own voice scratched my brain in the most satisfying way possible.
I was impressed by how realistic and natural it looked when I tried it myself. Native speakers told me in the comments that the translations were very good, and even “perfect” at times. But I should apologize to the people of Brazil for not using their flag on the Portuguese part as it was their dialect. Desculpe!
How you can try it out
Making this didn’t require any extreme editing and was much easier than I thought. I used HeyGen’s Video Translate tool to make it happen.
Step 1: Go to HeyGen’s Video Translate tool and make an account
When I created my account, I got 2 free credits. Each credit lets you translate a one-minute video. So you can try this out without getting your wallet out.
Step 2: Upload a video and select the target language
They only allow videos up to 5 minutes long. Once you’ve uploaded your video, select the target language you want to translate it to.
Available languages (as of September 21): English, Spanish, French, Italian, Hindi, German, Polish, Portuguese, Mandarin, Japanese, Dutch, Turkish, and Korean.
Step 3: Just wait
Once you submit your video, all you have to do is wait. When I tried my first video, it took a few hours for it to be complete. The processing time is faster if you upgrade to a paid plan. Either way, HeyGen will send you an email once the translation is complete.
What this means for the future of content and media 🔮
This type of technology has applications far and wide. If you can imagine all the types of video content we consume or interact with, you start to realize the massive potential:
Movies
TV Shows
YouTube videos
Online courses
Interviews
Livestreams
The main benefit that binds them all is the power to lower language barriers and make content more accessible to a wider audience. There’s an abundance of insightful, entertaining, and heartwarming video content that we don’t consume because it’s not in our language. We can go beyond our content “bubble” and feel more immersed in foreign content with new perspectives and distinctive creative expressions.
An important question naturally arises: “What will happen to voice actors? Does this technology make them obsolete?”
It’s important to remember that I’ve only recorded a few seconds of myself talking in each language, and it would be premature to extrapolate this into a death sentence for the voice acting industry.
I believe this technology will impact people in different ways:
Helping small- and medium-sized creators lower language barriers at an affordable price
Today, not everyone who makes video content has the resources to hire voice actors and translate their movies, shows, or videos into multiple languages. At most, they create subtitles and that’s it. A prime example of this is the countless YouTube channels that are only available in one language.
The biggest channels, like MrBeast who has dubbed his videos in 11 different languages, can afford voice actors. But he’s an outlier in the ocean of small- to medium-sized YouTube channels run by a few people with limited resources.
These channels were never going to afford or invest time in hiring and recording voice actors for each language. But these new AI tools can open up a new door to dub videos at an affordable price. By lowering language barriers, they can now tap into an international audience and grow their channel/business to (maybe) reach MrBeast-level success.
Yesterday, YouTube announced that they’re bringing an AI-dubbing tool to creators along with other AI-powered tools. Read their announcement
The impact on Hollywood-tier movies and TV shows
Of course, we still need to talk about the impact on voice actors who are regularly hired to dub blockbuster movies and TV shows. Will they stop hiring voice actors and just replace them with AI tools?
If you’ve been reading Year 2049 for a while, you know that I don’t like blanket statements and generalizations because reality is complex and nuanced.
Those who do replace voice actors with AI tools might be disappointed, and even alienate some native speakers. Let’s take a look at some of the more critical comments on my video:
The translations are good but they seem to miss the mark in sounding natural. Some friends also told me that they felt like a machine was speaking. Maybe this will improve over time, but there’s still a need for human translators to use the right words and convey the right meaning and emotions when moving between languages.
If you asked me what the best solution to improving dubbed video content, I would say this:
Use human translators to translate the content because they can use words, expressions, and dialogue that sound natural to a native speaker
Use human voice actors that speak the language to record the audio because their accent sounds natural while maintaining the emotion/tone of the original video
Use AI to regenerate the video and map the dubbed voiceover to the original actor’s lips to make it seem more natural. The biggest complaint (probably) about dubbed videos is the mismatch between the audio and an actor’s lip movements.
If we can map a native speaker’s voice to the lips of the original actor, that would be the highest-quality output with this type of technology.
Like I’ve said multiple times before, humans and AI work better together than either of them alone.
Someone asking the smart questions: Terry
I couldn’t end today’s newsletter without including Terry’s insightful question… 🤣
The future is too exciting to keep to yourself
Share this post in your group chats with friends, family, and coworkers.
If a friend sent this to you, subscribe for free to receive practical insights, case studies, and resources to help you understand and embrace AI in your life and work.
What you may have missed
If you’re new here, here’s what else I published recently:
You can also check out the complete Year 2049 archive to browse through all previous case studies, insights, and tutorials.
How would you rate this week's edition?
For someone who speaks six languages starting from childhood it added German, French, Dutch,English, Spanish and Italian. I must say that the AI translations came across quite well. Without nit picking. All that one wants it to get the main meaning. My SO is Chinese and I don't speak Mandarin but being a French speaker I LOVE the sound of it. I always feel Mandarin is the French of Asia while Japanese is the Italian of Asia. I often am on the "Little Red Book" (the Chinese Tik Tok) because there is so much creativity and fun. What is astounding is the power of so many people working hard on getting traction. Creating whimsical new electric cars, motorcycles and the unlimited amount of Art and handicraft. I often comment in English and the barrier of language is so obvious.
Only few manage to reply in English. I wish there was a way to instantly translate what people say on it in Mandarin to English. I even post on in and add Google txt translations and yea I have 9 followers!