Aim is to ‘polarise’: Influencer deepfakes used to stoke Canada-US tensions
Sign up now: Get ST's newsletters delivered to your inbox
Content creators have reportedly had their identities stolen by a network using artificial intelligence to inflame the online conversation around crumbling ties between the US and Canada.
PHOTO: ADOBE STOCK
- McGill University researchers found nearly 300 Instagram and TikTok accounts using AI deepfakes of real creators to inflame Canada-US trade war tensions.
- The network posted thousands of videos, got about 20 million views, and used fake scripts and mixed political claims to “polarise” viewers.
- TikTok and Meta removed the accounts, but influencers like Jasmin Laine and Ron Butler said the deception caused harm and was hard to stop.
AI generated
MONTREAL - Jasmin Laine has hundreds of thousands of subscribers, but some videos of her discussing the Canada-US trade war contain words she says she never spoke.
Laine is one of hundreds of content creators who, according to researchers at McGill University, had their identities stolen by a network using artificial intelligence to inflame the online conversation around the crumbling ties between the North American neighbours.
“There you have your face and your voice and what looks like you speaking, in some cases, to something that you actually don’t stand for,” Laine told AFP.
The McGill Centre for Media, Technology and Democracy study published in mid-November identified nearly 300 Instagram and TikTok accounts misusing authentic faces in a coordinated operation which published thousands of videos and pulled in some 20 million views.
The personalities whose likenesses were appropriated ranged from Christian lifestyle influencers to political commentary accounts from both sides of the spectrum.
Synthetic, inaccurate and inflammatory online material has repeatedly targeted Canadians since the trade war began, but the researchers said the digitally altered footage of real creators made the network more difficult to detect.
Before the launch of TikTok’s new deepfake detection tool on Nov 20, the Canadian researchers found only 4% of the operation’s accounts triggered the platform’s synthetic content warning.
The study also reported that the network further used AI to generate the videos’ scripts, reposting those which garnered the most engagement.
Polarising content
Aengus Bridgman, one of the report’s authors and director of the Media Ecosystem Observatory, said the network appeared to originate outside of North America.
The researchers found certain accounts registered in China and saw content largely posted in the middle of the day in Asia.
There was no clear evidence of the content being monetised, but Bridgman said whether the effort sought commercial gain or was solely a propaganda campaign “the idea is to polarise.”
The videos displayed varying levels of accuracy – often disseminating outright falsehoods – and the accounts took inconsistent political stances. Some would blame Ottawa for the trade conflict while others pointed the finger at Washington.
Canadian users appeared to be unwittingly interacting with the videos without realising they were fake.
Laine, whose content leans Conservative and often criticises the approach of Canada’s ruling Liberal government, said she found herself defending against accusations of being paid to parrot talking points she never uttered because of the inauthentic content.
Bridgman said the accusations levied at the real influencers was a “very good example of harm” inflicted by the operation.
“They have fallen hook, line and sinker for the network content,” he said.
Platform saturation
A TikTok spokesperson told AFP that it had permanently banned the accounts mentioned in the report, along with several others, for violating its rules about misleading content.
“As tactics evolve, we continue to collaborate with researchers on this industry-wide challenge and strengthen our moderation technology,” the spokesperson said.
AFP also reached out to Meta about the report and while the company declined to comment it said it had taken down the implicated Instagram accounts.
Ron Butler, a Canadian real estate influencer who also had his identity stolen by the network, said this was not the first time he had been impersonated or misrepresented.
He expressed satisfaction at the network’s unmasking while also telling AFP he is resigned to the presence of deceptive online material.
“You can’t stop this because there’s too many people who can do it,” he said. “It’s too easy to do.”
Laine also accepted the risk of being misrepresented online but said she was frustrated to see the conversation about the trade war artificially inflamed.
“It just flattens it into these very typical, very divisive talking points that I just find are really unproductive and unhelpful.” AFP
