NEW YORK - The buzz nowadays is about generative AI, which can produce new content based on the patterns and structure of the data it is trained on.

But it is the next version of artificial intelligence development that gives Foreign Minister Vivian Balakrishnan pause.

The advent of more sophisticated AI with a transactional ability – the capacity to negotiate, transact, communicate, mobilise and interact with other AI agents, as well as with human beings – will have a profound impact on societies, politics and economy, he said, during a doorstop interview with Singapore media at the conclusion of his visit to the United Nations this week.

“Sometimes it will be impossible to know whether the entity that you’re negotiating with on the other side is an AI agent or human being,” he said.

This might have especially dangerous implications in the field of defence, he said, when AI is on the trigger and human intelligence is challenged by the sheer speed needed for decision-making.

“One impact this has is that it reduces dramatically the decision time for not just military commanders in the field, but even for commanders-in-chief, presidents and prime ministers to determine the appropriate strategic response.”

The unprecedented situation – where AI makes lethal decisions on its own – requires deliberation, but the path ahead is far from clear.

Both the United States and China are in the race to develop AI-controlled autonomous weapons, news reports say, and are not talking to each other about taking precautions.  

“If there is intense competition, it’s very hard to get all the players to sit down in the same room and arrive at the appropriate compromises and a level of transparency and good faith in order to generate these rules,” Dr Balakrishnan said.

“If we can get together... have enough transparency so that people know what the other side is doing, and they also know what precautions both sides can take... at least you avoid accidental wars or unnecessary escalations.”

Then you can get a somewhat safer world, he said, drawing a parallel with the Cold War era, when the idea of mutually assured destruction from nuclear weapons was horrendous enough to act as a deterrent.

On the other hand, if the contest promotes innovations and is part of a single shared ecosystem of technologies, this might be another period of significant progress, he noted.

“These are early days yet, the conversations are just beginning. 

“But I’m glad that it’s happening on a multilateral global stage because these are challenges that cannot be solved just by one country alone, or by one bloc on its own,” Dr Balakrishnan said.

The next two to five years will be critical, he predicted.

“We will get a clearer indication of which way the world is going –  towards a more dangerous use of these frontier technologies, or towards a more collaborative, productive and safer use of these new technologies,” said Dr Balakrishnan.