From Fortune:

The creator of an audio deepfake of US President Joe Biden urging people not to vote in the New Hampshire primary this week has been suspended by ElevenLabs. ElevenLabs’ technology was used to make the deepfake audio, according to Pindrop Security Inc.

Once the deepfake was traced to its creator, that user’s account on ElevenLabs was suspended. The company has stated that it is dedicated to preventing the misuse of audio AI tools and takes any incidents of misuse extremely seriously. ElevenLabs itself announced an $80 million financing round from investors including Andreessen Horowitz and Sequoia Capital earlier this week.

The fake robocall of Biden has alarmed disinformation experts and elections officials alike, as it illustrates the relative ease of creating audio deepfakes. The New Hampshire Attorney General’s office has opened an investigation into the incident. Researchers confirmed that the audio was almost certainly created with ElevenLabs’ technology, though the company’s speech classifier couldn’t detect its own audio unless it’s analyzing the raw file.

Technology investors are a source of funding for multiple AI startups developing synthetic media, which will significantly transform the media and gaming industry. ElevenLabs CEO stated recently that his company is dedicated to content moderation, with the vast majority of their interactions are in the positive realm.



Read more: AI firm ElevenLabs bans account for Biden audio deepfake