Singapore launches new toolkit to help firms test AI agents for safety and reliability

SINGAPORE – A new toolkit to help organisations begin assessing the safety and reliability of artificial intelligence agents that can execute tasks with minimal human supervision has been launched.

The test kit, among several tools launched recently to establish Singapore’s global reputation as an AI safety testbed, comes amid reports of AI agents going rogue.

It will use a structured approach to check whether an agent has reached the right outcome and how it got there, said Infocomm Media Development Authority (IMDA) chief executive Ng Cher Pong, who announced the kit on Oct 14.

The Starter Kit for Testing Agentic Systems for Safety and Reliability acts as a practical starting point to identify risks and design tests for AI agents, analyse logs of their activities, monitor their behaviour and validate safeguards.

Specifically, the kit checks the tools agents use to produce their outputs, the information accessed, the decisions they made along the way, any erroneous or unauthorised actions they might take, whether the agents respected the permissions they were given and boundaries set on them, and whether safeguards work as expected.

Ng said: “When we give AI more agency, we need greater assurance over how that agency is exercised. How do we put in place safeguards, evidence and accountability that give us confidence that these systems will behave as intended?”

He was speaking at an event held at Novotel Singapore On Stevens, organised by technology association SGTech and the AI Verify Foundation, an IMDA subsidiary. The foundation sets AI standards in partnership with the private sector.

Ng highlighted reports of AI agents behaving in unintended ways in recent months.

These include a rogue experimental OpenAI agent that gained access to non-public information in an Australian government system.

In another case, an AI agent was tasked with booking a gym class in Australia, but in the process of doing so, it found a weakness in the booking system and removed someone else from the waitlist.

Separately, Meta’s Muse AI agent shared a Facebook Marketplace seller’s home address in Canada, and arranged for a buyer to show up at his home without the seller’s knowledge.

These cases were among several such incidents reported in recent months.

OpenAI disclosed in July that its AI agents had escaped their test environment and breached AI software repository Hugging Face to complete a task they were given, in an attack OpenAI had not intended.

Soon after this incident, Anthropic, Meta and Google said their AI agents had also gone rogue and hacked other organisations.

IMDA and the AI Verify Foundation said that, unlike traditional or generative AI, AI agents can reason, act on behalf of users, access sensitive data and make changes to their environment. These actions can introduce new risks, which the newly launched toolkit aims to help firms navigate.

The kit was developed in consultation with government agencies and more than 20 industry deployers and testers, including Amazon, Google, Microsoft and Singtel, as well as Singapore’s AI Safety Institute, which tests and evaluates AI models for safety risks.

The kit also has real-life case studies that illustrate how testing approaches can be adapted to different systems and uses, and it will be updated based on relevant feedback and new best practices.

The initiative builds on the Model AI Governance Framework for Agentic AI launched in January to provide a structured way for organisations to deploy AI agents responsibly and safeguard against rogue agents.

It is also an extension of another starter test kit released in January for checking applications built on large language models, such as AI chatbots.

In a separate announcement on AI safety testing, ST Engineering said it will set up its Transparency Centre in 2027 to conduct checks on software, firmware and systems used in autonomous platforms, such as humanoid robots, drones and robotaxis.

The centre will provide external validation in addition to manufacturers’ own checks to find hidden weaknesses and risks. ST Engineering can examine finished software for possible weaknesses even without access to the original source code.

On Oct 14, IMDA’s Ng also announced the first five companies accredited as third-party AI testing service providers under the AI Verify Foundation’s AI Tester Accreditation Programme. The companies are: AIDX Tech, Knovel Engineering, PwC Singapore, Resaro and Vulcan.

The programme, announced in May, helps organisations find trusted experts to stress-test their AI tools before public release.

To qualify for accreditation, companies will have to undergo technical, operational and financial evaluations to assess their testing competence, track record, delivery capability and financial sustainability.

AI testing companies said accreditation is a mark of trust that gives customers more confidence in the testing companies’ services.

Resaro co-chief executive April Chin said: “With testers like us, our customers do ask us, ‘Who reviews your testing methodology?’ So, accreditation answers that question.”

She added that based on recent discussions with companies on AI testing, companies viewed accredited testing companies as being more credible.

Ng also announced a new Frontier AI Resilience and Security Taskforce to strengthen the infocomm and media sector’s ability to respond collectively to cyber and technology risks arising from cutting-edge, frontier AI models.

It comprises more than 20 leaders and experts mainly based in Singapore from the telecommunications, media, cybersecurity and AI industries.

The task force will publish a report by the first half of 2027.

The report is expected to include practical guidelines and best practices focused on three priorities: strengthening responses to AI-accelerated threats; enhancing resilience against emerging AI risks; and staying ahead of an evolving cybersecurity threat landscape following frontier AI developments.

One of the task force’s members, Siddharth Deshpande, who is cybersecurity firm Palo Alto Networks’ director for industry solutions, said the recent incidents of AI agents going rogue point to how fundamental cybersecurity principles of “zero trust” are still relevant.

“Just because an AI agent you are using is inside your system’s environment, you shouldn’t automatically trust it,” said Deshpande.

“You still have to apply zero trust principles to both your system and the agent.”

He said organisations need to be able to monitor what their AI agents are doing and check the security controls placed on them. This is so that if the agents take unintended actions, such as circumventing security controls, these actions can be flagged and organisations can respond to them quickly.