Democrats call for investigation into faulty AI-generated intel report

Richard Blumenstein/US Air Force

A group of top Democratic senators is calling on government watchdogs to investigate a series of mistakes in US military targeting plans caused by artificial intelligence, after a CNN report detailed how one such error, according to a source familiar with the incident, “almost started a war.”

In a letter to Defense Secretary Pete Hegseth and Director of National Intelligence Jay Clayton sent on Friday, the senators cite CNN’s reporting on a close call with a Chinese ship in the Middle East as an example of recent events that “have given us growing concern about the extent to which agencies under your oversight have prioritized acceleration of AI capability adoption and ‘experimentation’ over effective governance.”

CNN reported Friday that a special operations command analyst relied on AI to generate an intelligence report that inaccurately identified the material on a Chinese cargo ship.

The report led the US military to believe the ship was transporting components of a nuclear weapons program, according to sources. The military began facilitating plans to intercept the vessel before officials realized, only just before the planned operation, that the report had been generated with the help of AI and was “entirely false.”

The kind of “ hallucination ” that the tool conjured has not been an isolated incident across the intelligence community since the use of AI began proliferating across government, one source familiar with the episode told CNN.

In the letter, Sens. Mark Warner, Jack Reed and Chris Coons called for an “immediate investigation by relevant Inspectors General, with a higher degree of public transparency in the ultimate findings, given the impacts these perceived failures have on public confidence in our intelligence and warfighting missions.”

The letter also references a deadly February US strike on a school in Minab, Iran, that killed nearly 200 children and adults. Three sources told CNN earlier this year the strike came after senior US military commanders bypassed warnings in critical databases, including one powered by AI, that intelligence about potential targets in Iran was severely out of date .

“Public reports that AI targeting platforms have repeatedly generated spurious outputs (based on outdated input data or outright hallucination) risk serious operational error, in addition to undermining the confidence of both the American public and international partners in the accuracy and reliability of U.S. intelligence and intelligence-informed kinetic action,” the senators wrote.

The senators’ letter demands Hegseth and Clayton “promptly investigate this matter” and provide relevant inspectors general with “unrestricted access to both instances this year in which media reports have suggested significant errors in AI-enabled targeting workflows, as well as any additional instances that may have so far not been publicly reported.”

The letter comes as the debate over the government’s role in implementing AI safeguards has come to dominate the halls of Congress. Democratic members have intensified their calls for regulation, while Republicans, including President Donald Trump and House Speaker Mike Johnson, have warned that slowing development may cost the US in the AI race with China .

At the Pentagon, Hegseth’s push to broadly infuse AI into military operations has not come without roadblocks. The Defense Department has sparred with Anthropic over the AI company’s refusal to remove internal safety guardrails that would prevent the military from using its Claude AI model in autonomous weapons and mass surveillance. Anthropic argued its models were not reliable enough to be used for such purposes.

理查德·布卢门斯坦/美国空军

一群民主党高级参议员呼吁政府监督机构调查美国军方人工智能造成的一系列目标选择错误。此前，CNN的一篇报道详细描述了其中一起错误，据一位知情人士透露，该错误“几乎引发了一场战争”。

在周五发给国防部长皮特·赫格塞斯和国家情报总监杰伊·克莱顿的一封信中，参议员们引用了 CNN 关于在中东与中国船只险些相撞的报道，作为近期事件的一个例子，这些事件“让我们越来越担心，在你们的监督下，各机构在多大程度上优先考虑加速人工智能能力的采用和‘实验’，而不是有效的治理”。

CNN周五报道称，一名特种作战司令部分析员依靠人工智能生成了一份情报报告，该报告错误地识别了一艘中国货船上的货物。

据消息人士透露，这份报告使美军误以为该船正在运输核武器项目的部件。军方开始着手制定拦截计划，但就在行动即将开始之际，官员们才意识到这份报告是由人工智能生成的，而且“完全是捏造的”。

一位熟悉此事的消息人士告诉 CNN，自从人工智能在政府部门广泛应用以来，这种工具产生的“幻觉”在情报界并非个例。

在信中，参议员马克·沃纳、杰克·里德和克里斯·库恩斯呼吁“相关监察长立即展开调查，并提高最终调查结果的公开透明度，因为这些被认为的失误会对公众对我们的情报和作战任务的信心产生影响。”

信中还提到了今年2月美国对伊朗米纳布一所学校的致命袭击，那次袭击造成近20​​0名儿童和成人丧生。今年早些时候，三位消息人士告诉CNN，此次袭击发生之前，美国高级军事指挥官无视了关键数据库（包括一个由人工智能驱动的数据库）发出的警告，这些警告表明有关伊朗潜在目标的情报严重过时。

参议员们写道：“有公开报道称，人工智能目标平台反复生成虚假输出（基于过时的输入数据或完全的幻觉），这不仅会造成严重的操作失误，还会削弱美国公众和国际伙伴对美国情报和基于情报的军事行动的准确性和可靠性的信心。”

参议员们的信中要求赫格塞斯和克莱顿“立即调查此事”，并向相关监察长提供“不受限制的访问权限，以查阅今年媒体报道中暗示人工智能驱动的目标定位工作流程存在重大错误的两个案例，以及任何其他可能尚未公开报道的案例”。

这封信发出之际，关于政府在实施人工智能安全保障措施方面应扮演何种角色的辩论已成为国会的主要议题。民主党议员加大了对监管的呼吁力度，而包括唐纳德·特朗普总统和众议院议长迈克·约翰逊在内的共和党人则警告说，放缓人工智能的发展速度可能会让美国在与中国的人工智能竞赛中处于劣势。

在五角大楼，赫格塞斯推动人工智能广泛应用于军事行动的努力并非一帆风顺。国防部与人工智能公司Anthropic就其拒绝移除内部安全防护措施一事争执不下，这些措施旨在阻止军方将其Claude人工智能模型用于自主武器和大规模监控。Anthropic辩称，其模型的可靠性不足以用于此类用途。