SINGAPORE – Generative artificial intelligence (AI) models like ChatGPT are typically trained on data from the open Internet.

But doing so might not adequately represent Singapore’s cultural context, such as its mix of racial groups and languages.

To ensure AI models developed here are well suited to the Republic, the Infocomm Media Development Authority (IMDA) announced on Tuesday that it has set up a collaborative experimental platform that AI developers can use to ensure algorithms are built responsibly.

Dubbed the Generative AI Evaluation Sandbox for Trusted AI, the programme will contain standardised evaluation tests to guide companies in setting up guard rails to prevent their systems from committing errors or showing bias.

Findings from these tests will contribute to a guide that contains recommendations for AI models developed here, taking into account cultural sensitivities that developers should be mindful of.

Announcing the programme at the Singapore Week of Innovation and Technology (Switch) conference at Marina Bay Sands on Tuesday, Deputy Prime Minister Heng Swee Keat said the initiative is the first of its kind, and will give AI developers the tools needed for their own assessment of AI models.

He said: “Critically, the sandbox will equip (AI) app developers with the skills and methodologies to conduct generative AI evaluation. Today, these capabilities reside largely with AI model developers.”

“This new sandbox will help all stakeholders develop a better understanding of AI safety and risk mitigation, and work together to address the safety issue that is of global concern,” he added.

Mr Heng encouraged companies to explore generative AI, noting that Enterprise Singapore and IMDA will also partner with trade association SGTech to convene a panel of industry experts to recommend relevant solutions for enterprises.

IMDA said the sandbox initiative will put AI models to the test in various fields, such as human resources and security, to expose gaps in the way AI is currently assessed.

During assessment, participating companies can undergo a process called “red-teaming”, which is to subject a system to a series of rigorous tests to find gaps in safety, added IMDA. Using this method, a large language model can be probed with multiple prompts to test if it will produce an undesired answer.

Language models can be tested for their ability to weed out toxic content, and whether they show bias towards certain demographics, political views and subjective opinions.

Where appropriate, developers will also be urged to check if their systems can explain how they make decisions and whether they are trained to rope in humans when necessary, according to the programme’s first draft to evaluate AI models.

Tech giants such as Microsoft, Google, IBM and Amazon Web Services have joined the programme. 

An IMDA spokesman told The Straits Times: “Large language models today are trained on Internet data, which may not be representative of the nuances of Singapore’s cultural context. For example, in terms of knowledge understanding, it may not appreciate that within racial groups, there is a diversity of faiths and languages.”  

The spokesman added: “As cultural evaluation is a nascent area, we work with model developers to develop a methodology to identify and weed out these concerns in the models in a systematic manner, which could also be applied to other countries besides Singapore.”

The programme falls under IMDA’s AI Verify Foundation, a coalition of more than 60 tech firms that was announced in June to tackle pressing issues in AI such as bias, copyright and its susceptibility to making up lies.

The foundation will discuss AI standards and best practices, and create a neutral platform for collaborating on governing AI.

As part of the initiative, IMDA rolled out an AI toolkit that lets firms check their AI systems for bias and potential leaks for free.

Organised by trade agency Enterprise Singapore, the Switch trade show is expected to host about 15,000 attendees, including entrepreneurs, government agencies and investors, from more than 100 countries.

Some 350 exhibitors from around the world have set up booths, with a focus on AI, healthcare, sustainability and other tech innovations.