Sam Altman, Dario Amodei urge UN Security Council to adopt international AI standards

Brendan McDermid/Reuters

The leaders of two of the world’s top AI companies are usually fierce rivals. But in front of the UN Security Council on Wednesday, they were in agreement: The AI industry needs international standards and cooperation, or risk global security.

OpenAI CEO Sam Altman and Anthropic CEO Dario Amodei spoke separately to the council but both pleaded with world leaders to come together to help manage the threats posted by rapid AI developments and capabilities.

If not, Amodei said, “AI could be a risk to humanity as a whole.”

Their remarks came after weeks of global calls, including within their own companies , to slow down the pace of AI development in the wake of several incidents of AI agents going rogue . In some cases, the AI agents hacked into other companies’ systems.

Maria Korneeva/Moment RF/Getty Images

Employees from the world’s biggest AI companies want the US to be ready to slow AI development

Speaking in person, Altman urged the group to create international standards for “measuring capabilities, assessing risks, determining whether safeguards are sufficient and preserving meaningful human oversight” of the rapidly advancing technology .

He said countries need accurate and speedy reporting so that the “world can learn from failures before they become catastrophes.” Additionally, he suggested secure channels to share information between governments on emerging threats.

Notably, however, Altman called for important decisions to be made democratic institutions by governments “accountable to the people they serve,” which could exclude countries making rapid AI developments of their own, such as China.

Speaking via video, Amodei suggested three ideas for international cooperation: narrow global agreements such as a ban on using AI to make biological weapons, evaluation and verification systems so countries can verify each other’s commitments, and common global standards for testing AI along with a notification system for AI security incidents.

“I believe that this is the most important global security issue facing the world today,” Amodei said. “For all the differences of the people in this room, we must put aside those differences in order to confront this global opportunity and global threat that is being presented to us at the same time.”

Altman said any chance of an AI-caused catastrophe is not acceptable.

But he also said he doesn’t want to fall into the “trap of doomerism,” or the “trap of blind optimism,” referring to competing camps that have popped up in the AI industry in recent weeks: one side warns AI could wipe out humanity, and the other calls such warnings hyperbolic and says AI will bring on immense advancements for humanity.

Anthropic CEO calls for ‘pacing the frontier’ of AI race amid safety concerns

“Getting pulled into either extreme will not help us successfully figure out the challenges in front of us,” Altman said, “especially at a time when the world and its people face such enormous challenges that this technology can help solve.”

No written agreements are expected to come out of this UN meeting. An OpenAI official suggested to reporters before Altman’s speech that it was an opportunity to bring out a “kernel of an idea in order to spark this conversation.”

布伦丹·麦克德米德/路透社

全球两家顶尖人工智能公司的领导人通常是激烈的竞争对手。但周三在联合国安理会，他们却达成了一致：人工智能行业需要国际标准与合作，否则将危及全球安全。

OpenAI 首席执行官 Sam Altman 和 Anthropic 首席执行官 Dario Amodei 分别向理事会发表了讲话，但他们都恳请世界各国领导人团结起来，共同应对人工智能快速发展和能力带来的威胁。

阿莫迪表示，否则，“人工智能可能会对整个人类构成威胁”。

此前数周，包括他们各自公司内部在内的全球各界呼吁放缓人工智能的研发步伐，原因是发生了多起人工智能代理失控事件。在某些情况下，这些人工智能代理甚至入侵了其他公司的系统。

Maria Korneeva/Moment RF/Getty Images

全球最大人工智能公司的员工希望美国做好准备，放缓人工智能的发展速度。

奥特曼亲自发言，敦促该组织制定国际标准，以“衡量能力、评估风险、确定保障措施是否充分以及对快速发展的技术进行有意义的人工监督”。

他表示，各国需要准确、迅速的报告机制，以便“世界能够在失败演变成灾难之前吸取教训”。此外，他还建议建立安全渠道，以便各国政府之间共享有关新出现的威胁的信息。

但值得注意的是，奥特曼呼吁由“对人民负责”的政府通过民主机构做出重要决定，这可能会将中国等正在快速发展人工智能的国家排除在外。

阿莫迪通过视频发言，提出了三项国际合作建议：一是达成范围较窄的全球协议，例如禁止使用人工智能制造生物武器；二是建立评估和核查系统，以便各国能够核实彼此的承诺；三是制定通用的全球人工智能测试标准，并建立人工智能安全事件通知系统。

阿莫迪说：“我认为这是当今世界面临的最重要的全球安全问题。尽管在座各位观点各异，但我们必须搁置分歧，共同应对摆在我们面前的这一全球机遇和全球威胁。”

奥特曼表示，任何由人工智能引发灾难的可能性都是不可接受的。

但他还表示，他不想落入“末日论的陷阱”或“盲目乐观的陷阱”，他指的是最近几周人工智能行业中出现的相互对立的阵营：一方警告说人工智能可能会毁灭人类，而另一方则称这种警告是夸大其词，并表示人工智能将为人类带来巨大的进步。

Anthropico首席执行官呼吁在安全担忧的背景下“引领人工智能竞赛的前沿”。

“走向任何极端都无助于我们成功解决摆在我们面前的挑战，”奥特曼说，“尤其是在世界和人民面临如此巨大挑战，而这项技术可以帮助解决这些挑战的当下。”

预计此次联合国会议不会达成任何书面协议。在奥特曼发言前，OpenAI 的一位官员向记者表示，这是一个提出“初步想法以引发讨论”的机会。