Spotify is making big moves on AI, testing AI translation of popular podcasts to different languages and confirming it will not ban AI-generated music.
Spotify CEO Daniel Ek talked all things AI in conversation with the BBC, saying the streamer will not ban AI-generated music.
Ek says there are limits to what is acceptable, though, and claims “we have a very large team that is working on exactly these types of issues”.
One red line is AI impersonating a real artist, which was established earlier this year when an AI-generated track purporting to come from Drake and Abel Tesfaye, also known as The Weeknd, was uploaded to platforms including Spotify.
The track was removed, but not before it became a viral sensation.
“You can imagine someone uploading a song, claiming to be Madonna, even if they’re not. We’ve seen pretty much everything in the history of Spotify at this point with people trying to game our system,” Ek told the BBC.
Music that emulates the style of artists without flat-out impersonating them gets a pass. But there’s a blurry line here. Will AI-generated content have to directly claim a link to a real artist to get in trouble?
How Spotify uses AI to translate podcasts
Spotify has announced testing of AI-derived translations of podcasts, making select English language shows accessible in Spanish, French, and German.
The streamer plans to trial the technique with popular podcasts including Armchair Expert, The Bill Simmons Podcast, and Diary of a CEO.
OpenAI’s technology will be used for the translation. It’s a tool called Whisper, which was announced in September 2022. Whisper was trained on 680,000 hours of content, according to the OpenAI website, “collected from the web”.
You can already hear it in action. Spotify CEO Daniel Ek tweeted a demo reel of snippets.
Podcasters - what if I told you could offer your pod to any listener around the world, in their own local language but still keep it in your own voice? That’s the pilot we’re launching @Spotify!
— Daniel Ek (@eldsjal) September 25, 2023
It’s called Voice Translation and using AI, translates podcasts episodes into… pic.twitter.com/kYq0bgxJYq
You can also listen to full episodes over at Spotify’s Voice Translation Hub, which is where these translated episodes will be collated during the pilot programme.
This method is much cheaper than getting a voice actor to overdub podcasts in different languages, and maintains the character and timbre of the podcasters’ voices.
Unlike some earlier demos of this AI tech in action, Spotify’s implementation does not also provide lip-sync. But that makes perfect sense when Spotify delivers podcasts in audio, rather than video.
