Making deepfakes gets cheaper and easier, thanks to AI

NEW YORK – It would not be completely out of character for Mr Joe Rogan, the comedian turned podcaster, to endorse a “libido-boosting” coffee brand for men.

But when a video circulating on TikTok recently showed Mr Rogan and his guest, Mr Andrew Huberman, hawking the coffee, some eagle-eyed viewers were shocked – including Mr Huberman.

“Yep that’s fake,” Mr Huberman wrote on Twitter after seeing the advertisement, in which he appears to praise the coffee’s testosterone-boosting potential, even though he never did.

The ad was one of a growing number of fake videos on social media made with technology powered by artificial intelligence (AI).

Experts said Mr Rogan’s voice appeared to have been synthesised using AI tools that mimic celebrity voices. Mr Huberman’s comments were ripped from an unrelated interview.

Making realistic fake videos, often called deepfakes, once required elaborate software to put one person’s face onto another’s.

But now, many of the tools to create them are available to everyday consumers – even on smartphone apps, and often for little to no money.

The new altered videos – mostly, so far, the work of meme-makers and marketers – have gone viral on social media sites like TikTok and Twitter.

The content they produce, sometimes called “cheapfakes” by researchers, work by cloning celebrity voices, altering mouth movements to match alternative audio and writing persuasive dialogue.

The videos, and the accessible technology behind them, have some AI researchers fretting about their dangers, and have raised fresh concerns over whether social media companies are prepared to moderate the growing digital fakery.

Disinformation watchdogs are also steeling themselves for a wave of digital fakes that could deceive viewers or make it harder to know what is true or false online.

“What’s different is that everybody can do it now,” said assistant professor of library and information science Britt Paris of Rutgers University, who helped coin the term “cheapfakes”.

“It’s not just people with sophisticated computational technology and fairly sophisticated computational know-how. Instead, it’s a free app.”

Reams of manipulated content have circulated on TikTok and elsewhere for years, typically using more homespun tricks like careful editing or the swopping of one audio clip for another.

In one video on TikTok, United States Vice-President Kamala Harris appeared to say that everyone who had been hospitalised for Covid-19 was vaccinated.

In fact, she said the patients were unvaccinated.

Graphika, a research firm that studies disinformation, spotted deepfakes of fictional news anchors that pro-China bot accounts distributed in late 2022, in the first known example of the technology being used for state-aligned influence campaigns.

But several new tools offer similar technology to everyday Internet users, giving comedians and partisans the chance to make their own convincing spoofs.

Last month, a fake video circulated showing US President Joe Biden declaring a national draft for the war between Russia and Ukraine.

The video was produced by the team behind Human Events Daily, a podcast and live stream run by Mr Jack Posobiec, a right-wing influencer known for spreading conspiracy theories.

In a segment explaining the video, Mr Posobiec said his team had created it using AI technology.

A tweet about the video from The Patriot Oasis, a conservative account, used a breaking-news label without indicating the video was fake.

The tweet was viewed more than eight million times.

Many of the video clips featuring synthesised voices appeared to use technology from ElevenLabs, an American start-up co-founded by a former Google engineer.

Last November, the company debuted a speech-cloning tool that can be trained to replicate voices in seconds.

ElevenLabs attracted attention in February after 4chan, a message board known for having racist and conspiratorial content, used the tool to share hateful messages.

In one example, 4chan users created an audio recording of an anti-Semitic text using a computer-generated voice that mimicked that of actress Emma Watson.

Motherboard reported earlier on 4chan’s use of the audio technology.

ElevenLabs said on Twitter that it would introduce new safeguards, like limiting voice cloning to paid accounts and providing a new AI detecting tool.

But 4chan users said they would create their own version of the voice-cloning technology using open source code, posting demos that sound similar to audio produced by ElevenLabs.

Experts who study deepfake technology suggested that the fake ad featuring Mr Rogan and Mr Huberman had most likely been created with a voice-cloning programme, though the exact tool used was not clear.

The audio of Mr Rogan was spliced into a real interview with Mr Huberman discussing testosterone.

The results are not perfect.

Mr Rogan’s clip was taken from an unrelated interview posted last December with Mr Fedor Gorst, a professional pool player.

Mr Rogan’s mouth movements are mismatched with the audio, and his voice sounds unnatural at times.

If the video convinced TikTok users, it was hard to tell: It attracted far more attention after it was flagged for its impressive fakery.

TikTok’s policies prohibit digital forgeries “that mislead users by distorting the truth of events and cause significant harm to the subject of the video, other persons or society”.

Several of the videos were removed after The New York Times flagged them to the company. Twitter also removed some of the videos.

Federal regulators have been slow to respond.

One federal law from 2019 requested a report on the weaponisation of deepfakes by foreigners, required government agencies to notify Congress if deepfakes targeted elections in the United States, and created a prize to encourage research on tools that could detect deepfakes.

“We cannot wait for two years until laws are passed,” said postdoctoral researcher Ravit Dotan, who runs the Collaborative AI Responsibility Lab at the University of Pittsburgh.

“By then, the damage could be too much. We have an election coming up here in the US. It’s going to be an issue.” NYTIMES