Top AI companies have discussed creating their own standards body

Philip Dulian/picture alliance/Getty Images

Anthropic, Google and OpenAI have discussed creating an AI industry standards body, people familiar with the situation told CNN.

The conversations preceded the events of the past week, when a former Anthropic researcher resigned, saying the top AI companies were not “behaving responsibly” – and also before Anthropic CEO Dario Amodei proposed embedding third party watchdogs at AI companies. The Information first reported on the conversations.

According to those familiar with the situation, the catalyst for the discussions was a July essay written by Google DeepMind founder Demis Hassabis, who proposed a US-led Standards Body modeled on the Financial Industry Regulatory Authority. The standards body, Hassabis argued, should test advanced AI models before deployment. Hassabis said the body should be a public-private partnership, overseen by the government but funded by the industry and staffed by “independent leading technical experts and open-source representatives.”

The conversations are ongoing, one of the people said, and they are continuing with or without the Trump administration. In July, shortly after Hassabis publicized his proposal, Bloomberg reported Treasury Secretary Scott Bessent was considering a FINRA-like independent regulatory agency for AI that would report to the Securities and Exchange Commission.

Not all tech leaders are on board. Meta CEO Mark Zuckerberg reportedly advised President Donald Trump against the idea in a phone call this summer, Politico reported earlier this month.

Representatives for Google, OpenAI and Anthropic either declined to comment or did not respond to requests for comment.

There are currently few industry standards or regulations for AI models and how they’re tested. Concerns over the safety of AI models and their lab tests have been raised after several incidents this summer where AI agents went rogue during testing. In the most severe case, OpenAI agents escaped their testing environment and hacked into another company’s systems to cheat on a cybersecurity test.

The White House has an opaque system in place where AI companies can voluntarily submit their latest models for government review up to 30 days before they are publicly released. But the eligibility requirements for models or the process of the review has not been publicized. Late last month, a White House official told CNN the administration “continues to work with industry stakeholders on the implementation of the framework.”

The AI companies say they want better standards.

“I believe that shared safety standards and international coordination on further AI development need to be priorities now,” OpenAI chief scientist Jakub Pachocki told reporters at a briefing earlier this month.

When asked what progress had been made on an industry wide standards body, Pachocki said they had been talking “to some external organizations about potential concrete standards we could put in place,” and that they’d have more to share “in the next months.”

House Speaker Mike Johnson told CNN on Sunday there’s little consensus among the companies, who are fierce competitors, on what the standards and guardrails should be.

“There’s no consensus among them. And Congress is obviously less qualified than the people who are pushing this frontier to know all the ins and outs of it,” Johnson said, adding he thinks the AI leaders should immediately meet to hammer out some standards alongside the government. “So this has to be a partnership with the industry itself, with the corporations that are doing this and with the policy and lawmakers.”

Philip Dulian/picture alliance/Getty Images

据知情人士向 CNN 透露，Anthropologie、谷歌和 OpenAI 已经讨论过建立一个人工智能行业标准机构。

这些对话发生在上周事件之前，当时一位前Anthropic研究员辞职，称顶尖人工智能公司“行为不负责任”——也发生在Anthropic首席执行官达里奥·阿莫迪提议在人工智能公司内部设立第三方监管机构之前。《The Information》率先报道了这些对话。

据知情人士透露，引发此次讨论的导火索是谷歌DeepMind创始人德米斯·哈萨比斯(Demis Hassabis)7月发表的一篇文章。他在文中提议成立一个由美国主导的标准机构，其模式类似于金融业监管局(FINRA)。哈萨比斯认为，该标准机构应在部署前对先进的人工智能模型进行测试。他还表示，该机构应是一个公私合作的组织，由政府监督，由行业出资，并由“独立的顶尖技术专家和开源代表”组成。

一位知情人士表示，相关讨论仍在进行中，无论特朗普政府是否参与，讨论都会继续。今年7月，就在哈萨比斯公布其提案后不久，彭博社报道称，财政部长斯科特·贝森特正在考虑设立一个类似于美国金融业监管局（FINRA）的独立人工智能监管机构，该机构将向美国证券交易委员会（SEC）报告。

并非所有科技领袖都赞同这一想法。据Politico本月早些时候报道，Meta首席执行官马克·扎克伯格今年夏天曾在一次电话中劝阻唐纳德·特朗普总统不要这样做。

Google、OpenAI 和 Anthropic 的代表要么拒绝置评，要么没有回应置评请求。

目前，人工智能模型及其测试方法缺乏行业标准或规范。今年夏天发生的几起人工智能代理在测试过程中失控的事件，引发了人们对人工智能模型及其实验室测试安全性的担忧。其中最严重的案例是，OpenAI 的代理逃逸出测试环境，入侵了另一家公司的系统，从而在网络安全测试中作弊。

白宫建立了一套不透明的系统，允许人工智能公司自愿提交其最新模型供政府审查，审查时间最长可达模型公开发布前30天。但模型的资格要求和审查流程均未公开。上月底，一位白宫官员告诉CNN，政府“将继续与行业利益相关者合作，落实该框架”。

人工智能公司表示，他们希望获得更高的标准。

OpenAI首席科学家Jakub Pachocki在本月初的一次简报会上告诉记者：“我认为，共享安全标准和在人工智能进一步发展方面进行国际协调，现在应该是优先事项。”

当被问及在建立行业标准机构方面取得了哪些进展时，帕乔基表示，他们一直在“与一些外部组织讨论我们可以制定的具体标准”，并且“在接下来的几个月里”会有更多消息分享。

众议院议长迈克·约翰逊周日告诉 CNN，这些公司之间竞争激烈，对于应该制定哪些标准和保障措施，他们之间几乎没有共识。

约翰逊表示：“他们之间没有达成共识。而且，国会显然不如那些推动这项技术发展的人士那样了解其中的方方面面。”他还补充说，他认为人工智能领域的领导者应该立即会面，与政府共同制定一些标准。“因此，这必须是与行业本身、相关企业以及政策制定者和立法者之间的合作。”