AllttA, a band with just about 97,000 monthly listeners on Spotify, had on March 30 released a track titled Savages, which quickly became the talk of the town because of billionaire rapper Jay-Z’s involvement in the track.

The 53-year-old American – who has more than 35 million monthly listeners on Spotify – did not collaborate with the band, but the song featured his vocals prominently. Rapper and songwriter J. Medeiros had used an artificial intelligence (AI) filter to mimic Jay-Z’s voice.

The song sparked a debate on the potential capabilities of such technology – already seen in the likes of ChatGPT – and the implications for creatives and lawmakers alike.

“Watching this @AllttaMusic experiment get 1m+ views in an hour and reading all the comments like: ‘I’m afraid!’ ‘This is great!’ ‘This is an abomination!’ ‘Where’s the law!’ ‘Take this down!’ ‘Put it on Spotify!’ Makes me feel like I’m making actual art… let’s go!” said Medeiros on the day of release in a now-deleted tweet.

American audio engineer and record producer-executive Young Guru – whose real name is Gimel Androus Keaton – posted an excerpt of the song in an Instagram post on March 31, calling for action.

“I’ve been trying to tell everyone that this is where we are now with AI. For some reason, this one got everyone’s attention,” wrote Mr Keaton on Instagram.

“On the one hand, I’m well aware that you can’t stop technology. Once the genie is out of the box, you (can’t) put him back in. On the other hand, we have to protect the rights of the artiste. Not only the artiste but everyone in society,” he continued, adding that a person’s name, image and likeness should not be able to be taken without permission.

Mr Keaton said that the only way he sees of dealing with the issue is to “learn from past mistakes” and “change the law”.

“You would be a fool to chase every person that is going to do this,” he said, citing the case of controversial peer-to-peer file-sharing application Napster, which was throttled with lawsuits over copyright infringement and piracy before it was shut down in 2001, two years after its release in 1999.

“We could change the United States law tomorrow, but the Internet is worldwide. What a time we live in,” he added.

As of April 3, Jay-Z has not issued a statement about the AI-generated vocals in Savages.

More than 1,000 AI experts and industry executives – including Tesla founder Elon Musk – called in an open letter in February for a six-month pause in training of powerful AI systems, citing potential risks to society and humanity.

“Powerful AI systems should be developed only once we are confident that their effects will be positive and their risks will be manageable,” said the open letter, which also detailed potential risks to society and civilisation from human-competitive AI systems in the form of economic and political disruptions.

The letter called on developers to work with the regulatory authorities and policymakers on governance, with the suggested pause to last until shared safety protocols for such designs were developed, implemented and audited by independent experts.

Ethics of recreating a voice

In 2021, Roadrunner, a documentary about late chef Anthony Bourdain, sparked controversy after its director revealed that Bourdain’s voice had been recreated using AI.

This occurred for a total of about 45 seconds and in three separate instances, including one where an e-mail that he sent to a friend was read aloud in his voice.

Critics and commenters debated the ethics of the move.

“Maybe if I’m running for president, I can use Anthony Bourdain as my speaker,” Dr Sameer Singh, an associate professor of computer science at the University of California, Berkeley, told TheWrap.

“He can introduce me before every speech. Maybe that’s not something Anthony Bourdain would do, but who has the right to allow that?”

In February, Mr Keaton had voiced his concern about AI voice filters in a now-deleted post on Instagram with a viral clip showing a man creating a “fake” Kendrick Lamar rap track.

“I’m at the point where I can voice my concerns with our current state of AI. I have followed as many versions of what AI could do for some years now,” pop culture website Complex cited Mr Keaton as saying.

In his post, he recounted being shown an AI project that could “analyse and create music in any style of any musician”.

“I didn’t think we would get here this fast with the voice,” he said, adding that while AI technology is impressive, it raises many questions for artistes.

“Of course, my mind goes to the ethical and legal aspects of what can be done,” he added. “You add that to the power of ChatGPT and you realise we are in a very ground-breaking but dangerous moment.”

Mr Keaton said “it’s not the tech, it’s the evil that men do with the tech” and that there are legal aspects “because at this present moment, you can’t copyright a voice”.

He raised a potential scenario: “There are still states that don’t even have a law against revenge porn. I’m sorry to go there but imagine the repercussion on our kids when, five years from now, some high-school kid gets mad at his ex-girlfriend and creates a whole ‘deep fake’ that sounds and looks real.

“Imagine the political landscape where we can’t believe anything we hear because someone will claim he didn’t say it.”

Using AI voice filters for good?

In 2022, blockbuster Top Gun: Maverick, starring Tom Cruise as daredevil pilot Pete “Maverick” Mitchell, actor Val Kilmer made a cameo appearance as fellow pilot Tom “Iceman” Kazansky, whom he had played in the original film.

Initial reports incorrectly claimed that Kilmer’s lines in the film were entirely AI-generated, as he had lost his voice following treatment for throat cancer and now speaks using a electronic voice box.

However, Top Gun: Maverick director Joseph Kosinski told USA Today that the technology was not needed, with Kilmer’s voice “digitally altered and blended a little bit just for clarity”.

Kilmer revealed in August 2021 that he had partnered with Spotify-owned AI voice platform Sonantic to recreate his old speaking voice.

To do so, he supplied Sonantic with archival footage of his voice that was fed into the company’s algorithms and cleaned up to create the model, reported news website TheWrap.

Sonantic chief executive Zeena Qureshi told TheWrap that other documentary film-makers and companies had approached them asking about AI voice models following the release of Roadrunner, which Sonantic did not work on.

Using AI, Sonantic has created computer-generated voices that are either completely synthetic, or mimic the voice of a real person. Its voices have been used in video games, Hollywood sound production and speech therapy.

In a Sonantic video on YouTube, Kilmer said that after he lost his voice, people struggled to understand him when he spoke.

“Despite all that, I still feel I’m the exact same person, still the same creative soul. A soul that dreams ideas and stories constantly,” he said.

“But now I can express myself again, I can bring these dreams to you, and show you this part of myself once more. A part that was never truly gone, just hiding away.”

He thanked the team at Sonantic in a separate statement for restoring his voice in a way he had “never imagined possible”.

“As human beings, the ability to communicate is the core of our existence, and the side effects from throat cancer have made it difficult for others to understand me. The chance to narrate my story, in a voice that feels authentic and familiar, is an incredibly special gift.”