How ElevenLabs enabled AI-powered dubbing for PM Modi’s Lex Fridman interview

Prime minister Narendra Modi‘s chat with MIT scientist and American podcaster Lex Fridman made headlines for a lot of causes. The prime minister made feedback supporting US president Donald Trump, lauded Elon Musk and his mission with the Division of Authorities Effectivity (DOGE), and way more. He additionally spoke about artificial intelligence (AI) and what it can enable.

Apparently, the know-how, now indispensable in any sector, was working its magic in that interview itself.

The podcast was dubbed by voice-based generative AI startup ElevenLabs. Utilizing AI voice cloning know-how, the corporate supplied genuine voices of PM Modi and Fridman in Hindi and English, even attaining their intonations, feelings, and magnificence of speech.

The audio and subtitles of the podcast are presently accessible in Hindi, English, and Russian, and extra languages will quickly be added. The unique mixture of Hindi and English can be accessible for individuals who want to hear it.

Mati Staniszewski, cofounder of ElevenLabs, stated in a publish on X, “Impressed by Lex Fridman and Narendra Modi’s dialog – so happy with our workforce for making it accessible in each English and Hindi. ElevenLabs was based on the again of poor-quality dubbing. Seeing it evolve into an immersive expertise that preserves emotion and intent is unimaginable.”

Discover the stories of your interest

What is AI voice cloning?

AI voice cloning is a technology that uses artificial intelligence to replicate a person’s voice. Essentially, it creates a digital copy of someone’s voice that can then be used to generate speech.

This has many applications, including creating character voices for video games and animations, personalised voice assistants or customer service tools for businesses, and dubbing videos into different languages in the original speaker’s voice.

Big India market

Siddharth Srinivasan, a senior executive at the firm, had told ET earlier this month that India has emerged as the most important person marketplace for the US-based startup, with Hindi being among the many high three languages.

The $3.3 billion AI audio firm, backed by Silicon Valley-based enterprise capitalists a16z and Sequoia, selected India as its first worldwide enlargement market final September and is investing to localise language content material.

India is a voice-first market with over 145 million customers, which might maintain voice enter rising, stated Srinivasan. In the meantime, on the output aspect, India may need over 100 million podcast listeners over the subsequent two years, whereas AI brokers are anticipated to turn out to be a $500 million alternative by 2030, he stated.