Anthropic says it blocked possible attempts to use AI to develop bioweapons

Chris Ratcliffe/Bloomberg/Getty Images

Anthropic says it has blocked multiple accounts that used the company’s artificial intelligence models in ways that could support the development of biological weapons.

In a report published Thursday, the AI company known for its Claude model said it considers biological misuse one of the “most serious risks” to artificial intelligence models.

The new report outlines five real-life case studies in which users “circumvented controls” that block users from specific regions and “engaged in other efforts to obfuscate the purpose of their research to evade our safeguards.” The examples include possible gain-of-function research and involve both infectious diseases, such as bird flu, and novel venoms and toxins.

Looking over 30 days of activity, Anthropic said, it identified about 35 “distinct research efforts” with potentially concerning activity. However, it acknowledged that it could not be certain whether the actors “intended harm” or were conducting research for legitimate scientific purposes. Anthropic said individuals in the case studies were “working scientists” but did not identify the research institutions or countries where the activities took place.

“Sophisticated attacks no longer require sophisticated attackers,” the new report says. “The cybersecurity skills of AI models means that AI has collapsed the labor and tooling gap that used to separate well-resourced, state-sponsored operations from individual operators.”

Jonathan Raa/NurPhoto/Getty Images

‘Gambling with our lives’: Another AI employee quits over safety concerns

One of the case studies detailed in the report involved a request for assistance writing a grant application for funding of gain-of-function research on the transmissibility and immune evasion of the chikungunya virus.

In another case, a researcher outside the US was using Claude for work on bird flu that focused on viruses’ adaptation to mammals and how it caused severe illness.

The report also details cases on research into orthopoxviruses – a group that includes variola, which causes smallpox, and mpox – as well as venom toxins.

Drug development is considered one of the most promising and optimistic applications of AI; leaders in the industry often point to its potential to cure cancer and other diseases. But the same features that make AI powerful for discovering new materials and medications to help people could also make it useful for developing bioweapons in the hands of bad actors.

Anthropic said that it wants its models to be useful for scientific research, but the latest models have become capable of much more complex work than earlier versions — and the potential for misuse rises with the potential benefits. It added that bad actors could try to use the beneficial biological uses of AI to “maintain a kind of ‘plausible deniability’” about potentially harmful research.

“As our models become increasingly capable, approaching or exceeding expert performance at challenging scientific tasks, we expect their impact will only increase, both in beneficial and potentially harmful contexts,” the Anthropic report said. “As Al models become more widely used, providers will continue to acquire threat-relevant visibility into real-world use that even governments and intergovernmental organizations lack.”

The company said that recent models, including Claude Fable 5, have “stronger safeguards that restrict access to a wide range of dual-use biological research queries.”

Former Anthropic researcher warns AI could ‘kill us all’

Anthropic said that there’s evidence that the safeguards it has in place are working but that even more robust safety systems will be necessary as the AI models and possible threats become more sophisticated. The company also noted in the report that it disrupted efforts to use its models for surveillance operations, scams and the development of conventional weapons like drones and missiles.

The disclosure follows a growing chorus of warnings from AI employees who’ve recently left their companies over safety concerns. Some shared concerns that the technology is moving too quickly and will one day outpace humans’ ability to control it — and called for governments to consider how to slow down development if things get out of hand.

“If you extrapolate into the future, the level of capabilities of these AIs … they could cause extreme havoc,” Jacob Coxon, a former Anthropic employee, told CNN’s Anderson Cooper on Wednesday.

In a resignation thread on X this week, the 27-year-old AI researcher said “the people building AI earnestly believe that it could kill us all by the end of the decade.”

Chris Young/The Canadian Press/AP/File

Godfather of AI: Brace for more rogue AIs

Speaking Wednesday, Coxon said it could hack critical infrastructure — or build “extinction-level bioweapons.”

“There’s a lot of ways that the AI could actuate itself in the world,” he said.

In July, nearly 1,400 AI company employees signed an open letter urging the US government to regulate the technology to rein in Big Tech and slow the pace of AI to ensure its safety. And this week, OpenAI Chief Scientist Jakub Pachocki warned that AI capabilities are advancing faster than researchers’ ability to reliably monitor and control them.

For now, AI companies are largely policing themselves. The Trump administration has worked to undermine state AI regulations, and Congress has so far been unwilling to rein in the technology.

“We believe the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models,” Anthropic said in a statement Wednesday.

CNN’s Clare Duffy contributed to this report.

克里斯·拉特克利夫/彭博社/盖蒂图片社

Anthropic公司表示，他们已经封禁了多个利用该公司人工智能模型进行生物武器研发的账户。

这家以 Claude 模型而闻名的 AI 公司在周四发布的一份报告中表示，生物滥用是人工智能模型面临的“最严重风险”之一。

这份新报告列举了五个真实案例，在这些案例中，用户“绕过了”针对特定地区用户的控制措施，并“采取其他手段来混淆其研究目的，以规避我们的安全保障措施”。这些案例包括可能的功能增益研究，涉及传染病（例如禽流感）以及新型毒液和毒素。

安特罗皮克组织表示，经过对30天活动的审查，他们发现了约35项“不同的研究活动”，这些活动可能令人担忧。然而，该组织承认，他们无法确定这些活动参与者是否“意图造成伤害”，或者他们开展研究的目的是出于合法的科学目的。安特罗皮克组织称，案例研究中的参与者是“科研人员”，但并未透露这些活动发生的机构或国家。

这份新报告指出：“复杂的攻击不再需要复杂的攻击者。人工智能模型的网络安全技能意味着，人工智能已经消除了过去将资源雄厚的国家支持的行动与个体操作者区分开来的劳动力和工具差距。”

Jonathan Raa/NurPhoto/Getty Images

“拿我们的生命做赌注”：又一名人工智能员工因安全顾虑辞职

报告中详细介绍的一个案例研究涉及请求协助撰写拨款申请，以获得对基孔肯雅病毒的传播性和免疫逃避能力进行功能获得性研究的资金。

在另一个案例中，一位美国境外的研究人员利用 Claude 进行禽流感研究，重点是病毒对哺乳动物的适应以及它如何导致严重疾病。

该报告还详细介绍了正痘病毒（包括引起天花的痘病毒和痘病毒）以及毒液毒素的研究案例。

药物研发被认为是人工智能最有前景的应用领域之一；业内领军人物经常指出其在治愈癌症和其他疾病方面的潜力。然而，人工智能之所以能够发现造福人类的新材料和药物，其强大的功能也可能被不法分子利用，用于开发生物武器。

人智公司表示，他们希望其模型能为科学研究做出贡献，但最新模型的功能已远超早期版本——潜在的益处越大，滥用的可能性也越高。该公司还补充说，不法分子可能会利用人工智能在生物学领域的有益应用，来掩盖其潜在的有害研究，从而“维持某种程度上的‘合理否认’”。

“随着我们的模型能力不断提升，在应对高难度科学任务时达到甚至超越专家水平，我们预计其影响只会越来越大，无论是在有益的方面还是在潜在的有害方面，”人类学研究所的报告指出。“随着人工智能模型的广泛应用，供应商将继续获得与威胁相关的、针对实际应用情况的洞察力，而这种洞察力甚至连政府和政府间组织都缺乏。”

该公司表示，包括 Claude Fable 5 在内的最新型号具有“更强大的安全保障，可限制对各种两用生物研究查询的访问”。

前人类学研究员警告称，人工智能可能“毁灭我们所有人”

安特罗皮克公司表示，有证据表明其现有的安全措施行之有效，但随着人工智能模型和潜在威胁日趋复杂，更强大的安全系统必不可少。该公司还在报告中指出，其已成功阻止了利用其模型进行监视、诈骗以及研发无人机和导弹等常规武器的企图。

此前，越来越多的AI员工因安全隐患而离职，他们纷纷发出警告。此​​次披露正值这一警告声浪日益高涨之际。一些员工担忧这项技术发展过快，终有一天会超出人类的控制能力，并呼吁各国政府考虑如何减缓其发展速度，以防事态失控。

“如果你把这些人工智能的能力水平推算到未来……它们可能会造成极大的破坏，”前人类学研究所员工雅各布·考克森周三告诉 CNN 的安德森·库珀。

本周，这位 27 岁的 AI 研究员在 X 论坛上发表了一篇辞职声明，称“开发 AI 的人们真心相信，到本十年末，AI 可能会杀死我们所有人”。

克里斯·杨/加拿大通讯社/美联社/档案照片

人工智能教父：准备迎接更多失控的人工智能

科克森周三表示，它可以入侵关键基础设施，或者制造“灭绝级别的生物武器”。

他说：“人工智能可以通过很多方式在世界上发挥作用。”

今年7月，近1400名人工智能公司员工签署了一封公开信，敦促美国政府对这项技术进行监管，以约束大型科技公司并减缓人工智能的发展速度，确保其安全性。本周，OpenAI首席科学家雅库布·帕乔基警告称，人工智能能力的提升速度已经超过了研究人员对其进行可靠监控和控制的能力。

目前，人工智能公司主要依靠自我监管。特朗普政府一直致力于削弱各州的人工智能监管，而国会迄今为止也不愿对这项技术进行限制。

“我们相信，如果业界能够采用合法、可验证的方式来共同努力，控制我们发布强大型号的速度，那么世界将会受益，”Anthropic周三在一份声明中表示。

CNN的克莱尔·达菲对本报道亦有贡献。