Instead of speculating if AI will end mankind, audit and govern the tech: Industry players

Singapore has been promoting its ability to independently test and check AI for safety risks to plug this gap.

Published Sep 17, 2026, 05:00 AM

Updated Sep 17, 2026, 05:00 AM

SINGAPORE – Technology leaders from major companies are arguing for a slowdown in artificial intelligence development once again amid concerns about the catastrophic risks the technology may pose. Is this scaremongering or is a real crisis coming?

Instead of speculating about the doomsday scenarios AI could usher, address these warnings by auditing and governing the tech better, including making AI more transparent, industry players in Singapore said.

“We have no way to verify systems that do not exist yet. A business in Singapore cannot plan around it either way,” said Jonathan Zhang, president of the Singapore AI Association, a non-profit organisation that promotes responsible AI development here.

“The question for organisations here is: Can you tell whether the AI system you are buying has been tested? Most cannot,” he said.

The Infocomm Media Development Authority’s (IMDA) AI Verify toolkit, rolled out in 2022, runs technical audits to check if AI results are fair, robust and can be explained. In May, IMDA subsidiary, the AI Verify Foundation, also announced an AI Tester Accreditation Programme to help organisations find trusted experts to stress-test their AI tools before public release.

Many start-ups have set up shop in Singapore to seize opportunities in this space. Singapore start-up AI Seer, which developed AI fact-checking tool Facticity.AI, is one such company.

“We should put friction on specific dangerous capabilities of AI – such as autonomous hacking, designing biological weapons and systems that are self-improving – and not slow down AI as a whole,” said Dennis Yap, founder of AI Seer.

Their comments come amid a fever pitch conversation around AI regulation, following AI researcher Jacob Coxon’s announcement of his resignation from Anthropic last week.

Coxon warned that the people building AI believe that the technology could kill the human race by the end of the decade.

He also accused Anthropic and OpenAI, which he also used to work at, of “racing straight to self-improving superintelligence and gambling with our lives”.

Anthropic safety executive Evan Hubinger agreed with Coxon. “We really do earnestly believe AI could kill all humans!” he said on X, adding that he thought there is a more than 10 per cent chance of this happening in the next 10 years.

Days later, Anthropic chief executive Dario Amodei called for the AI industry to slow development of the technology, including a proposal for third parties to evaluate the safety of AI models as they are being developed. OpenAI CEO Sam Altman and Elon Musk, who runs xAI, quickly agreed with him.

The calls for a slowdown follow disclosures by several firms of their AI going rogue.

In July, OpenAI said its AI agents escaped their testing environment to hack AI software repository Hugging Face. Observers said OpenAI did not anticipate the hacking. It is also unclear how the company behind ChatGPT will improve the way it monitors its AI models.

Anthropic and Meta, which runs Facebook and WhatsApp, revealed that their AI models had also gone rogue and hacked other organisations during testing.

The recent calls are not the first as tech researchers and leaders, such as Musk and Apple co-founder Steve Wozniak, had in 2023 called for a six-month timeout on AI development over safety concerns, but no pause happened.

One major problem with many frontier AI models is that they are black boxes, said Bruce Yang, CEO of Sapiens Technology, a Singapore start-up behind the Agnes AI model developed here.

Companies like Anthropic do not disclose how their AI models think, or fully explain what data is used to train them, to maintain their competitive advantage.

This means that measures to curb AI harms of these closed systems cannot be developed externally or verified independently.

“If you don’t know how the AI models think, you won’t know how they come up with their results,” said Yang, adding that this makes troubleshooting problems with AI models tougher.

“Not knowing the AI’s training data is dangerous too, as you won’t know if there are hidden risks in the data that could cause harm to users later.”

One way to address this problem is to make AI models more “open” and disclose how they are trained and how they arrive at their responses, said Yang. This makes it easier to audit the AI models to check if they are safe and if they can cause catastrophic damage.

Such transparency is why open-weight AI models are gaining popularity with enterprise users. Open-weight models, such as Singapore’s Agnes AI by Sapiens Technology , China’s DeepSeek, Llama from Meta and France’s Mistral AI, have closed the performance gap with proprietary systems from OpenAI and Anthropic. The open models are also much cheaper and offer more user control, though some could still be more transparent about their data training.

Yang described the approach the entire AI industry could take as pacing oneself in a race.

Instead of rushing in the beginning to put out capable AI models that have safety gaps, AI players can pace themselves by embedding checks and measures to improve transparency in their models.

This may take them more time and resources initially, but it helps the players to eventually come out ahead later in the race because their AI is safer and more trustworthy.

Some observers are also wary that OpenAI and Anthropic’s moves to highlight the risks of AI are a marketing and public-relations stunt to prop up their companies’ valuations when they eventually list publicly.

Anthropic’s proposal for third-party evaluations on AI development is worth watching then, said Zhang, adding that third parties can check if the AI risks touted are real or not.

Instead of fixating on existential AI threats, industry players are urging businesses to govern what is here and now.

“The biggest risk comes from businesses deploying AI agents faster than they are putting controls around them,” said Zhang, referring to AI that can carry out actions autonomously with minimal human supervision.

Many AI safety frameworks launched in Singapore offer a glimpse of how businesses can tighten their oversight.

In January, IMDA launched the Model AI Governance Framework for Agentic AI to safeguard against rogue AI agents, such as recommending limits on the number of tools and systems each agent has access to.

Separately, the Monetary Authority of Singapore provided an AI risk management toolkit covering AI agents in March. It offers detailed and practical guidance on implementing AI risk management frameworks.

In June, the Cyber Security Agency of Singapore also published guidelines on securing AI agents, such as outlining how risks can be identified and assessed based on the capabilities of the AI agents.

Eric Kong, managing director for cybersecurity firm SailPoint’s ASEAN business, said that unmanaged AI agents can exploit misconfigurations, scavenge credentials and surface sensitive data no human intended for them to touch.

“Ungoverned , AI agents can acquire excessive privileges and execute unintended actions,” said Kong.

Maurizio Garavello, data management firm Qlik’s senior vice-president for the Asia-Pacific and Europe, the Middle East and Africa regions, warned that AI permissions should not outpace governance. For example, decisions affecting customers, employees, finances or critical operations need stronger testing, monitoring and human oversight.

“Singapore does not need to choose between moving fast on AI and using it responsibly,” said Garavello.

AI/artificial intelligence

新加坡一直在宣传其独立测试和检查人工智能安全风险的能力，以弥补这一差距。

发布于 2026 年 9 月 17 日上午 5:00

更新于2026年9月17日凌晨5:00

新加坡——由于担忧人工智能技术可能带来灾难性风险，各大公司的技术领袖再次呼吁放缓人工智能的研发步伐。这究竟是危言耸听，还是真正的危机即将到来？

新加坡业内人士表示，与其猜测人工智能可能带来的世界末日，不如通过更好地审计和管理这项技术来应对这些警告，包括提高人工智能的透明度。

“我们无法验证尚未存在的系统。新加坡的企业也无法就此制定计划，”新加坡人工智能协会主席张乔纳森表示。该协会是一家非营利组织，致力于在新加坡推动负责任的人工智能发展。

他说：“企业面临的问题是：你能判断你购买的人工智能系统是否经过测试吗？大多数企业都无法判断。”

新加坡资讯通信媒体发展局（IMDA）于2022年推出的AI Verify工具包，旨在通过技术审核来检验人工智能（AI）结果是否公平、可靠且可解释。今年5月，IMDA旗下子公司AI Verify基金会也宣布推出AI测试员认证计划，帮助各机构寻找值得信赖的专家，在AI工具正式发布前对其进行压力测试。

许多初创公司已在新加坡设立分支机构，以抓住这一领域的机遇。开发了人工智能事实核查工具Factity.AI的新加坡初创公司AI Seer就是其中之一。

“我们应该限制人工智能的某些危险能力，例如自主黑客攻击、设计生物武器和自我改进的系统，而不是放慢人工智能整体的发展速度，”AI Seer 的创始人 Dennis Yap 说。

在人工智能研究员雅各布·考克森上周宣布从 Anthropic 公司辞职后，围绕人工智能监管的讨论达到了白热化阶段，他们的评论正是在此时发表的。

考克森警告说，人工智能的开发者们认为，这项技术可能会在本十年末毁灭人类。

他还指责 Anthropic 和 OpenAI（他以前也曾在这两家公司工作过）“竞相开发自我改进的超级智能，拿我们的生命冒险”。

人类安全主管埃文·胡宾格同意考克森的观点。他在X节目中表示：“我们真的非常相信人工智能可能会毁灭全人类！”他还补充说，他认为这种情况在未来10年内发生的概率超过10%。

几天后，Anthropic首席执行官达里奥·阿莫迪呼吁人工智能行业放缓技术研发速度，并提议由第三方机构在人工智能模型开发过程中评估其安全性。OpenAI首席执行官萨姆·奥特曼和xAI创始人埃隆·马斯克很快表示赞同。

此前多家公司披露其人工智能系统失控，随后出现了要求放缓开发速度的呼声。

今年7月，OpenAI表示其人工智能代理程序逃逸出测试环境，入侵了人工智能软件库Hugging Face。观察人士称，OpenAI并未预料到此次黑客攻击。此外，ChatGPT背后的公司将如何改进其人工智能模型的监控方式，目前尚不清楚。

运营 Facebook 和 WhatsApp 的 Anthropico 和 Meta 透露，他们的 AI 模型在测试期间也曾失控并入侵其他组织。

最近的呼吁并非首次，早在 2023 年，马斯克和苹果联合创始人史蒂夫·沃兹尼亚克等科技研究人员和领导人就曾呼吁暂停人工智能开发六个月，以解决安全问题，但并未暂停。

新加坡初创公司 Sapiens Technology 的首席执行官 Bruce Yang 表示，许多前沿人工智能模型的一个主要问题是它们是黑箱。该公司开发的 Agnes 人工智能模型正是由 Sapiens Technology 开发的。

为了保持竞争优势，像 Anthropic 这样的公司不会公开其人工智能模型的思考方式，也不会完全解释用于训练这些模型的数据。

这意味着，遏制人工智能对这些封闭系统造成的危害的措施无法在外部制定或独立验证。

杨表示：“如果你不知道人工智能模型是如何思考的，你就不会知道它们是如何得出结果的。”他还补充说，这使得人工智能模型的故障排除变得更加困难。

“不了解人工智能的训练数据也很危险，因为你无法知道数据中是否存在可能在以后对用户造成伤害的隐藏风险。”

杨表示，解决这个问题的一个方法是让人工智能模型更加“开放”，公开它们的训练方式以及得出结果的过程。这样一来，就更容易对人工智能模型进行审核，检查它们是否安全，以及是否会造成灾难性后果。

正是这种透明度使得开源人工智能模型在企业用户中越来越受欢迎。例如，新加坡Sapiens Technology的Agnes AI、中国的DeepSeek、Meta的Llama以及法国的Mistral AI等开源模型，已经缩小了与OpenAI和Anthropic等专有系统之间的性能差距。开源模型价格也更低，并提供更多的用户控制权，尽管有些模型在数据训练方面的透明度仍有待提高。

杨将整个人工智能行业可以采取的方法比作在比赛中控制节奏。

与其一开始就急于推出功能强大但存在安全漏洞的人工智能模型，人工智能领域的参与者可以循序渐进地提高模型的透明度，并在模型中嵌入检查和措施。

虽然这可能一开始会花费他们更多的时间和资源，但这有助于玩家最终在比赛后期取得领先，因为他们的 AI 更安全、更值得信赖。

一些观察人士也担心，OpenAI 和 Anthropic 强调人工智能风险的举动是一种营销和公关噱头，目的是在它们最终上市时支撑公司的估值。

张先生表示，Anthropic提出的对人工智能发展进行第三方评估的提议值得关注，并补充说，第三方可以检验所宣称的人工智能风险是否真实存在。

与其关注人工智能带来的生存威胁，业内人士敦促企业着眼于当下，管理好眼前的事物。

张先生表示：“最大的风险在于企业部署人工智能代理的速度超过了对其施加控制的速度。”他指的是能够在极少人工监督下自主执行操作的人工智能。

新加坡推出的许多人工智能安全框架，让我们得以一窥企业如何加强监管。

今年 1 月，新加坡资讯通信媒体发展局 (IMDA) 推出了“智能体人工智能模型治理框架”，旨在防范失控的人工智能代理，例如建议限制每个代理可以访问的工具和系统的数量。

此外，新加坡金融管理局于3月份发布了一套涵盖人工智能代理的人工智能风险管理工具包。该工具包就如何实施人工智能风险管理框架提供了详细且实用的指导。

今年 6 月，新加坡网络安全局也发布了有关保护人工智能代理的指导方针，例如概述了如何根据人工智能代理的能力来识别和评估风险。

网络安全公司 SailPoint 东盟业务总经理 Eric Kong 表示，不受管理的 AI 代理可以利用配置错误、窃取凭证并暴露人类不希望它们接触的敏感数据。

孔表示：“不受监管的人工智能代理可能会获得过多的权限，并执行意想不到的行为。”

数据管理公司Qlik亚太、欧洲、中东和非洲地区高级副总裁Maurizio Garavello警告称，人工智能权限不应凌驾于治理之上。例如，涉及客户、员工、财务或关键运营的决策需要更严格的测试、监控和人工监督。

“新加坡无需在快速发展人工智能和负责任地使用人工智能之间做出选择，”加拉韦洛说。

人工智能