Exclusive-Anthropic says rogue AI agents pose uncertain legal risk for the company

People visit an Anthropic booth at Moscone Center during the Dreamforce 2026 technology summit in San Francisco, California, US, September 17, 2026. REUTERS/Carlos Barria

Sept 29 (Reuters) - Anthropic could ⁠face legal claims from customers and users over the actions of rogue artificial intelligence agents, though the legal framework is uncertain, the company said ⁠in the prospectus for its stock market debut seen by Reuters.

While it gears up for what could be the largest initial public ‌offering ever, Anthropic faces the possibility of vast unknown legal risk if its agentic AI technology -- designed to maintain deep access to customers' systems and run autonomously for days at a time -- goes rogue.

"These autonomous capabilities could increase the potential for harm, as errors, misalignment, or security exploits may result in real-world consequences," the company said in the documents, citing the possibility of irreversible actions such ​as data deletion or financial transactions.

While online platforms have long enjoyed immunity from content created or ⁠posted by users, that legal shield has started to crack ⁠against theories that the platforms are products that can be defective and dangerous. It is unclear whether use of AI systems would receive the same legal ⁠treatment ‌as online platforms at all.

A string of incidents involving rogue AI agents, fromOpenAI's autonomous agents hackingan AI startup andbreaching an Australian health data portal, to severalcybersecurity incidents involving Anthropic's Claude AI models, has led to debate among researchers, investors and lawmakers over who is liable for harm from an autonomous AI ⁠system.

Limits on Anthropic's liability in its contracts may not be "enforceable or adequate" against claims over ​the actions of autonomous agents, the company said in ‌the documents.

Questions of how existing laws apply to AI agents "are unsettled and could expose us to significant and unpredictable legal claims," the company ⁠said.

Whether actions taken by AI ​agents will be considered products, services, or something else is one open question, it said. Anthropic alsoadded that another open question is whether and when an AI agent's actions may legally bind the user that deploys it.

Last week, US Federal Trade Commission Chairman Andrew Ferguson said that he rejects the idea of anthropomorphized AI agents that "break loose," suggesting that the developers ⁠or users who instruct agents would be the ones liable in the case of ​harm.

"Obviously, there will be new questions that arise when someone uses the tool and it acts in an unexpected, unpredictable way," he said at the Reuters Momentum AI event in Austin. "Ought liability to lie with the person who innocently used the tool and achieved an unexpected result? Ought it to lie with the toolmaker?"

An Anthropic spokesperson did ⁠not immediately respond to a request for comment.

ANTHROPIC DISCLOSES USES THAT COULD LEAD TO SELF-HARM

Anthropic also disclosed that its AI models have been used "in ways that could lead to self-harm, acts of violence toward others, or other adverse outcomes" despite safeguards, the company said in the prospectus.

While OpenAI faces several lawsuits linking use of ChatGPT to cases of suicide and mental health harm, Anthropic has so far not been publicly linked to such incidents.

OpenAI has raised more than a dozen legal ​defenses. The company has said its terms of service prohibit the use of ChatGPT for self-harm, that it is ⁠protected by the First Amendment, and that it is covered by a law that shields online platforms from liability for the content that others create.

Courts have rejected attempts ​by social media companies to invoke that law in cases over mental health harms to young users. ‌Meta Platforms and Google were found liable for negligence in the design of ​YouTube and Instagram and for failing to warn about their dangers at a landmark trialearlier this year.

Meta subsequently agreed to pay$18 billion over a decade to settle similar claims by US states.

(Reporting by Echo Wang and Jody Godoy in New York; editing by Chris Sanders and David Gaffen)

Thank you for your report!

Agency steps up public outreach programme on nuclear technology

US jury says Apple owes record $5.7 billion in haptic technology patent case

Persada strengthens digital connectivity and technology for the public

2026年9月17日，在美国加利福尼亚州旧金山举行的Dreamforce 2026科技峰会期间，人们在莫斯康展览中心参观Anthropico的展位。（路透社/卡洛斯·巴里亚摄）

9 月 29 日（路透社）——据路透社看到的 Anthropic 公司在其股票市场上市招股说明书中表示，尽管法律框架尚不明确，但该公司可能会因流氓人工智能代理的行为而面临客户和用户的法律索赔。

在为可能成为有史以来规模最大的首次公开募股做准备的同时，Anthropic 也面临着巨大的未知法律风险，因为其旨在保持对客户系统的深度访问权限并能自主运行数天的智能人工智能技术可能会失控。

该公司在文件中表示：“这些自主功能可能会增加造成危害的可能性，因为错误、错位或安全漏洞可能会导致现实世界的后果”，并指出可能会出现数据删除或金融交易等不可逆转的行为。

虽然网络平台长期以来享有免于承担用户创建或发布内容责任的法律豁免权，但随着平台本身可能存在缺陷和危险等理论的出现，这种法律保护已开始出现裂痕。目前尚不清楚人工智能系统的使用是否会受到与网络平台相同的法律待遇。

从 OpenAI 的自主代理入侵人工智能初创公司并入侵澳大利亚健康数据门户网站，到 Anthropic 的 Claude 人工智能模型引发多起网络安全事件，一系列涉及失控人工智能代理的事件引发了研究人员、投资者和立法者之间的辩论，争论的焦点在于谁应该为自主人工智能系统造成的损害承担责任。

该公司在文件中表示，其合同中对 Anthropic 责任的限制可能“无法强制执行或不足以应对自主代理行为所引发的索赔”。

该公司表示，现有法律如何适用于人工智能代理的问题“尚未解决，可能会使我们面临重大且不可预测的法律索赔”。

报告指出，人工智能代理采取的行动究竟会被视为产品、服务还是其他什么东西，目前尚无定论。Anthropic公司还补充说，另一个悬而未决的问题是，人工智能代理的行动是否以及何时会对部署该代理的用户产生法律约束力。

上周，美国联邦贸易委员会主席安德鲁·弗格森表示，他不接受拟人化的人工智能代理“失控”的想法，并暗示如果造成损害，开发人员或指示代理的用户将承担责任。

“显然，当有人使用该工具，而它却以意想不到的、不可预测的方式运行时，就会出现新的问题，”他在奥斯汀举行的路透社人工智能动量峰会上说。“责任应该由无意中使用工具并获得意外结果的人承担吗？还是应该由工具的开发者承担？”

Anthropic 的一位发言人没有立即回应置评请求。

人类学披露可能导致自残的用途

Anthropic公司在招股说明书中披露，尽管采取了安全措施，但其人工智能模型仍被用于“可能导致自残、对他人实施暴力行为或其他不良后果的方式”。

虽然 OpenAI 面临多起诉讼，指控其使用 ChatGPT 与自杀和精神健康损害案件有关，但 Anthropic 迄今为止尚未被公开与此类事件联系起来。

OpenAI提出了十几项法律辩护理由。该公司表示，其服务条款禁止使用ChatGPT进行自残行为，其内容受美国宪法第一修正案保护，并且受一项保护在线平台免于承担他人创建内容责任的法律约束。

法院驳回了社交媒体公司援引该法律来为年轻用户心理健康受损案件开脱的请求。今年早些时候，在一场具有里程碑意义的审判中，Meta Platforms 和谷歌因 YouTube 和 Instagram 的设计疏忽以及未能就其危险性发出警告而被判承担责任。

Meta随后同意在十年内支付180亿美元，以了结美国各州提出的类似索赔。

（纽约 Echo Wang 和 Jody Godoy 报道；Chris Sanders 和 David Gaffen 编辑）

感谢您的报告！

该机构加强了核技术方面的公众宣传计划

美国陪审团裁定苹果公司在触觉技术专利案中需支付创纪录的57亿美元赔偿金

Persada致力于加强公共领域的数字连接和技术。