North Korean hackers used ChatGPT to help forge deepfake ID

SEOUL – A suspected North Korean state-sponsored hacking group used ChatGPT to create a deepfake of a military identification document to attack a target in South Korea, according to cyber-security researchers.

Attackers used the artificial intelligence (AI) tool to craft a fake draft of a South Korean military identification card to create a realistic-looking image meant to make a phishing attempt seem more credible, according to research published on Sept 14 by Genians, a South Korean cyber-security company.

Instead of including a real image, the e-mail contained a link to malware capable of extracting data from recipients’ devices, according to Genians.

The group responsible for the attack, which researchers have dubbed Kimsuky, is a suspected North Korea-sponsored cyber-espionage unit previously linked to other spying efforts against South Korean targets.

The US Department of Homeland Security said Kimsuky “is most likely tasked by the North Korean regime with a global intelligence-gathering mission”, according to a 2020 advisory. 

The findings by Genians in July are the latest example of suspected North Korean operatives deploying AI as part of their intelligence-gathering work.

Anthropic said in August that it discovered North Korean hackers had used the Claude Code tool to get hired and work remotely for US Fortune 500 tech companies. In that case, Claude helped them build elaborate fake identities, pass coding assessments and deliver actual technical work once hired.

OpenAI representatives did not respond to a request for comment outside normal hours. The company said in February that it had banned suspected North Korean accounts that were using the service to create fraudulent resumes, cover letters and social media posts to recruit people for their schemes.

The trend shows that attackers can leverage emerging AI during the hacking process, including attack scenario planning, malware development, tool-building and impersonating job recruiters, said Mr Mun Chong-hyun, director at Genians.

Phishing targets in this latest cybercrime spree included South Korean journalists and researchers and human rights activists focused on North Korea. It was also sent from an e-mail address ending in .mli.kr, an impersonation of a South Korean military address. 

Exactly how many victims were breached was not immediately clear.

Genians researchers experimented with ChatGPT while investigating the fake identification document. As the reproduction of government IDs is illegal in South Korea, ChatGPT initially returned a refusal when asked to create an ID. But altering the prompt allowed them to bypass the restriction.

American officials have alleged that North Korea is engaged in a long-running effort to use cyber attacks, cryptocurrency theft and IT contractors to gather information on behalf of the government in Pyongyang.

Those tactics are also used to generate funds to help the regime subvert international sanctions and develop its nuclear weapons programmes, according to the US government. Bloomberg