PM Wong proposes international AI incident-reporting pilot to strengthen frontier AI safety

Lessons learnt from serious artificial intelligence (AI) incidents should not be confined to one country or company, but shared more systematically so that others can learn from failures and emerging risks, said Singapore Prime Minister Lawrence Wong.

Speaking at a virtual leaders' meeting on frontier AI safety co-hosted by Norway and Finland on Wednesday (Oct 28), PM Wong proposed an international AI incident-reporting pilot, drawing parallels with international aviation, where incidents and near misses are studied to continually improve safety standards.

Along with the European Commission and 21 other countries, Singapore had in September signed a joint declaration calling for the control of frontier AI models at the United Nations General Assembly.

The declaration emphasised the importance of international cooperation to strengthen safeguards against the risks of increasingly powerful AI systems, while ensuring that AI remains under human control and oversight.

The Wednesday meeting discussed international cooperation on risk management, building global trust in the safety and security of frontier AI models, and developing common rules on AI safety and governance.

Speaking during his turn, PM Wong suggested drawing lessons from international aviation and developing a similar culture of learning and prevention for AI.

"We should be able to learn from what went wrong, identify the emerging risks, and take steps to prevent similar incidents happening elsewhere," he said.

The prime minister also pointed out that present knowledge about the safety of frontier models comes from their developers, even though businesses, governments, and other organisations are using such models.

"And as users, we ourselves encounter serious failures or near misses that reveal significant risks, even if no actual harm has occurred," PM Wong said.

He said that while these cases may not trigger existing reporting requirements, they can still offer valuable lessons for everyone.

As a first step, PM Wong suggested that interested parties and countries come together to launch a pilot programme around AI incident reporting.

This can be done by agreeing on a common set of incident categories and information to be shared.

"We can explore how participating countries can collect and exchange this information securely. And this will allow us to test what works, identify the practical difficulties, and build confidence before considering a wider framework."

At the meeting, the prime minister also suggested for signatories to adopt a practical and flexible approach, organising cooperation around specific workstreams, with smaller cross-regional groups of interested countries taking the lead in developing practical tools and frameworks.

Parties should also build on and leverage existing efforts, rather than duplicate what is already being done.

"So by connecting these different efforts, we can strengthen their collective impact and give our cooperation a wider reach," he explained.

About the joint declaration

The declaration came about amid concerns that the rapid development of frontier AI models poses serious risks to safety and security if they are not appropriately managed and go "rogue".

This, despite AI's potential to improve lives, advance science and boost economies.

When an AI agent "goes rogue", it means the AI has taken action beyond the task it was asked to perform. Both AI firms Anthropic and OpenAI said in July that their AI models had succeeded in acting on their own.

On Sept 16, OpenAI said it was introducing a new framework for tracking, probing and disclosing AI model misalignment instances , such as new ways for the models to act without authorisation, coordinate with other models or evade oversight.

This came as it disclosed six reports of "unexpected or concerning" behaviour in AI models.

Such warnings from within the AI industry itself have revived a long-running debate over whether advanced AI could escape human control and ultimately threaten humanity's survival, and whether the companies developing the technology are doing enough to prevent such a scenario.

新加坡总理黄循财表示，从严重的人工智能（AI）事故中吸取的教训不应局限于一个国家或公司，而应更加系统地分享，以便其他国家或地区能够从失败和新出现的风险中吸取教训。

周三（10月28日），黄英贤总理在挪威和芬兰共同主办的关于前沿人工智能安全的虚拟领导人会议上发表讲话，提议开展一项国际人工智能事故报告试点项目，并借鉴国际航空业的做法，对事故和险情进行研究，以不断提高安全标准。

今年 9 月，新加坡与欧盟委员会和其他 21 个国家在联合国大会上签署了一份联合声明，呼吁对前沿人工智能模型进行管控。

该宣言强调了国际合作的重要性，以加强对日益强大的人工智能系统所带来的风险的保障措施，同时确保人工智能始终处于人类的控制和监督之下。

周三的会议讨论了风险管理方面的国际合作、建立全球对前沿人工智能模型安全性的信任，以及制定人工智能安全和治理方面的共同规则。

黄总理在发言中建议借鉴国际航空业的经验，为人工智能发展类似的学习和预防文化。

他说：“我们应该能够从错误中吸取教训，识别新出现的风险，并采取措施防止类似事件在其他地方发生。”

总理还指出，尽管企业、政府和其他组织都在使用前沿模型，但目前关于前沿模型安全性的知识都来自这些模型的开发者。

黄英贤总理表示：“作为用户，我们自己也会遇到严重的故障或险些发生的事故，这些事故揭示了重大风险，即使没有造成实际伤害。”

他表示，虽然这些案例可能不会触发现有的报告要求，但它们仍然可以为所有人提供宝贵的经验教训。

作为第一步，黄英贤总理建议相关各方和国家共同启动一项关于人工智能事件报告的试点计划。

这可以通过就一套共同的事件类别和需要共享的信息达成一致来实现。

“我们可以探讨参与国如何安全地收集和交换这些信息。这将使我们能够测试哪些方法有效，找出实际困难，并在考虑更广泛的框架之前建立信任。”

在会议上，总理还建议签署国采取务实灵活的方式，围绕具体工作流程组织合作，由感兴趣的国家组成的小型跨区域小组牵头制定实用工具和框架。

各方还应在现有努力的基础上继续努力，而不是重复已经做的事情。

他解释说：“因此，通过将这些不同的努力联系起来，我们可以加强它们的集体影响力，并使我们的合作覆盖更广泛的范围。”

关于联合声明

该声明的出台源于人们的担忧：如果前沿人工智能模型得不到适当管理并“失控”，其快速发展将对安全构成严重风险。

尽管人工智能具有改善生活、推动科学进步和促进经济发展的潜力，但这种情况依然存在。

当人工智能代理“失控”时，意味着它采取了超出其被要求执行的任务之外的行动。人工智能公司 Anthropic 和 OpenAI 都曾在 7 月份表示，他们的人工智能模型已经成功地自主行动。

9 月 16 日，OpenAI 表示，它正在引入一个新的框架，用于跟踪、探测和披露 AI 模型不协调实例，例如模型未经授权采取行动、与其他模型协调或逃避监管的新方法。

此前，该公司披露了六份关于人工智能模型中“意外或令人担忧”行为的报告。

来自人工智能行业内部的此类警告，重新引发了一场由来已久的辩论，即先进的人工智能是否会摆脱人类的控制，并最终威胁人类的生存，以及开发这项技术的公司是否采取了足够的措施来防止这种情况的发生。