The Pentagon’s new autonomy command must heed lessons new and old

Defense Secretary Pete Hegseth announces plans to create a new combatant command, Autonomous Warfare Command, during a Sept. 30 speech at Marine Corps Base Quantico in Virginia. U.S. Navy / Petty Officer 1st Class Eric Brann

AUTOWARCOM will have a big task. Luckily, it’s not starting from scratch.

As Pentagon leaders launch their latest effort to accelerate the development and deployment of autonomous and robotic systems, they can draw on key lessons from recent breakthroughs and past attempts.

The planned Autonomous Warfare Command, slated to launch by next October, will promote “not a particular drone, but the ability to adapt in combat” against “realistic opposition,” according to a Sept. 30 memo from Defense Secretary Pete Hegseth, who added: “We do not have a decade to transition our procurement and doctrine.”

That’s because on today’s battlefields, effective adaptation to “realistic opposition” is something that happens in weeks, not the months and years it typically takes defense contractors to update designs or even issue software patches. Fortunately, the Pentagon isn’t starting from scratch.

Start with the mission, not the robot

Start with asking commanders and operators what they want from autonomy and robots, rather than trying to dazzle them with an experiment or demonstration that isn't relevant to what they need.

One example is Maven Smart System, an AI intelligence platform widely adopted by the U.S. military and NATO forces—and a model for getting emerging technology to the force faster. A noted 2024 report for CNAS wrote, “A clear, urgent, but broad problem definition permitted Maven to more directly meet user needs.”

So what do combatant commanders want now? Here’s Gen. Frank Donovan, commander of U.S. Southern Command, speaking in May at a Defense One event:

“Most of the systems we're looking at are primarily our domain awareness systems, and…We can match the robots to the environment whether it swims, it flies, it has feet, whatever it does.”

Donovan had just finished leading the Defense Autonomous Warfare Group , the special-operations effort to apply autonomy and robotics to missions. He discovered that robots were needed not just to replace humans in combat but to better understand what was happening in the places where boots weren’t on the ground.

“Our partners, who have the access and placement where they live, have to work in, in the rough terrain, the jungle, over the horizon, thousands of miles at sea.”

Build communicative autonomy

Donovan said most of his work at the Defense Autonomous Warfare Group was about getting people partners to talk about what information they needed from robotic systems and what information they wanted those systems robots to have.

“We don't talk about robots. We talk about the data environment, right? With the different data layers that we need at the very forward edge so our [special operations forces] and our conventional force teammates with a...cell phone they can actually plug into that data network, and whatever robot shows up with the capability, they can leverage it instantaneously.”

Autonomous systems will become even more important when network access is blocked by adversary electronic-warfare capabilities, said one former senior defense official who worked to field AI software to U.S. and allied forces.

“The next step is getting these systems to true autonomy that can handle the operating environment. In any conflict, GPS and comms will be denied and degraded. We see it today in Ukraine and the Middle East, and it's a certainty in any future fight,” they said.

Buy the brains and body separately

Another lesson comes from the auto industry, which has turned to external partners for its autonomy needs.

By contrast, the former senior official said, “The Department has mostly asked industry to self-organize to deliver autonomy alongside the vehicle.” For example, Pentagon buyers seeking autonomous drones have simply asked manufacturers to build autonomous versions of their remote-controlled aircraft.

That’s the wrong approach, the former official said. Drone makers “build excellent platforms, but autonomy is its own discipline. Commercial self-driving figured this out years ago: the automakers build great vehicles and partner with autonomy companies for the driver,” they said. “Defense needs the same model.”

The former official added that Pentagon buyers should specify that the autonomy package should be modular—that is, able to be upgraded as threats change and technology improves.

That’s true not just for individual weapons, but across the force and indeed across alliances, they said. “Open standards are what make that partnership work at scale.”

This was underscored by a recent NATO experiment in Portugal. In September, the alliance’s Centre for Maritime Research and Experimentation, or CMRE, wrapped up its biggest annual exercise: Robotic Experimentation and Prototyping with Maritime Unmanned Systems. This year’s edition featured a demonstration that showed what an Autonomous Warfare Command will look to replicate: robots receiving orders not from humans but from software.

“For the first time, we had materialized our vision for future operations. We had algorithms looking at what the mission was looking like and automatically giving tasks to drones,” said João Alves, a principal scientist at CMRE.

That wasn’t the result of a new sensor or piece of AI software but rather a policy change: partner militaries agreeing to adopt a standard for the systems that they were buying.

The biggest challenge any military office, command, or unit will face when using autonomous weapons is knowing when to rely on them.

That’s the big conclusion of a 2017 study of autonomy safety failures involving Patriot missile batteries.The most dramatic case was the 2003 downing of an F-18 by a Patriot in Iraq.

“The tactical director at the battalion command and control node gave the order, ‘Bring your launchers to ready.’ That directive was tantamount to an order to engage. But that was not what the tactical director intended,” Army engineering psychologist John Hawley wrote in the study.

Even with everything else in place—a specific mission, open standards, and shared data, the right training—the biggest challenge any command or unit faces when employing autonomy is “sustained high reliability in a complex and unpredictable operational setting,” Hawley concludes.

That same question—does this software program understand what I intend it to do?—has only grown more urgent with each new instance of an AI software agent breaking the law in service of a poorly defined user request.

If AUTOWARCOM is to set a course for the military, its leaders must recognize why previous efforts in autonomy succeeded or failed, and acknowledge that human trust can’t be automated.

NEXT STORY: US spy chief tapped as White House AI czar

美国国防部长皮特·赫格塞斯于9月30日在弗吉尼亚州匡蒂科海军陆战队基地发表讲话，宣布计划成立一个新的作战司令部——自主作战司令部。美国海军/一级军士长埃里克·布兰恩

AUTOWARCOM将面临艰巨的任务。幸运的是，它并非从零开始。

五角大楼领导人启动最新举措，加速自主和机器人系统的开发和部署，他们可以借鉴近期突破和过去尝试中的关键经验教训。

根据国防部长皮特·赫格塞斯9月30日的一份备忘录，计划于明年10月启动的自主作战司令部将“不推广某种特定的无人机，而是推广在战斗中适应”的能力，以对抗“现实的对手”。赫格塞斯还补充道：“我们没有十年的时间来转变我们的采购和作战理论。”

这是因为在当今战场上，有效应对“现实对抗”只需数周时间，而不是国防承包商通常需要数月甚至数年才能更新设计或发布软件补丁的时间。幸运的是，五角大楼并非从零开始。

首先要考虑任务，而不是机器人。

首先要询问指挥官和操作员他们希望从自主性和机器人中获得什么，而不是试图用与他们的需求无关的实验或演示来迷惑他们。

例如，Maven智能系统就是一个人工智能平台，已被美国军方和北约部队广泛采用，并成为将新兴技术更快应用于部队的典范。CNAS在2024年的一份报告中指出：“清晰、紧迫但又广泛的问题定义使Maven能够更直接地满足用户需求。”

那么，作战指挥官们现在想要什么呢？以下是美国南方司令部司令弗兰克·多诺万将军五月份在Defense One活动上的讲话：

“我们目前研究的大多数系统主要都是领域感知系统，而且……我们可以根据环境调整机器人，无论它是游泳的、飞的、有脚的，还是其他任何方式。”

多诺万刚刚结束了领导国防自主作战小组的工作，该小组负责将自主作战和机器人技术应用于特种作战任务。他发现，机器人不仅用于在战斗中取代人类，而且还能更好地了解地面部队缺席地区的情况。

“我们的合作伙伴虽然拥有工作地点和资源，但他们必须在崎岖的地形、丛林、地平线之外、数千英里之外的海上工作。”

培养沟通自主性

多诺万表示，他在国防自主作战小组的大部分工作是让各方合作伙伴讨论他们需要从机器人系统获得哪些信息，以及他们希望这些系统的机器人拥有哪些信息。

“我们不谈机器人，我们谈的是数据环境，对吧？我们需要在最前沿部署不同的数据层，这样我们的特种部队和常规部队的队友，只要有手机，就可以接入这个数据网络，任何具备相应能力的机器人都能立即利用它。”

一位曾参与向美国及其盟军推广人工智能软件的前高级国防官员表示，当敌方电子战能力封锁网络访问时，自主系统将变得更加重要。

他们表示：“下一步是让这些系统真正实现自主运行，以应对各种运行环境。在任何冲突中，GPS和通信都会被切断或降级。我们今天在乌克兰和中东地区就看到了这种情况，而且在未来的任何冲突中，这种情况都必然会发生。”

大脑和身体分别购买。

汽车行业也给我们带来了另一个教训，该行业为了满足其自动驾驶需求，转向了外部合作伙伴。

相比之下，这位前高级官员表示，“国防部主要要求业界自行组织，在提供车辆自主功能的同时，也提供自主系统。” 例如，五角大楼采购自主无人机的客户只是要求制造商生产其遥控飞机的自主版本。

这位前官员表示，这种做法是错误的。无人机制造商“打造了优秀的平台，但自动驾驶本身是一门独立的学科。商用自动驾驶多年前就找到了答案：汽车制造商制造出色的车辆，然后与自动驾驶公司合作开发驾驶员所需的技术，”他们说。“国防领域也需要采用同样的模式。”

这位前官员补充说，五角大楼的采购人员应该明确规定自主系统组件应该是模块化的——也就是说，能够随着威胁的变化和技术的进步而升级。

他们表示，这不仅适用于单个武器，也适用于整个部队乃至整个联盟。“开放标准是这种伙伴关系能够大规模运作的关键。”

北约近期在葡萄牙进行的一项试验凸显了这一点。今年9月，北约海上研究与试验中心（CMRE）结束了其年度最大规模的演习：海上无人系统机器人实验与原型开发。今年的演习重点展示了自主作战指挥系统未来将要实现的目标：机器人不再接受人类的指令，而是通过软件进行控制。

“我们首次将对未来行动的愿景变为现实。我们拥有能够分析任务情况并自动向无人机分配任务的算法，”CMRE首席科学家João Alves说道。

这并非新型传感器或人工智能软件带来的结果，而是政策变化的结果：伙伴国家的军队同意为他们购买的系统采用统一的标准。

任何军事部门、指挥部或部队在使用自主武器时面临的最大挑战是，何时应该依赖它们。

这是2017年一项关于爱国者导弹系统自主安全故障研究的主要结论。其中最引人注目的案例是2003年爱国者导弹在伊拉克击落一架F-18战斗机。

“营级指挥控制节点的战术指挥官下达了命令：‘让你们的发射器进入准备状态。’这条指令等同于交战命令。但这并不是战术指挥官的本意，”陆军工程心理学家约翰·霍利在研究中写道。

霍利总结道，即使其他一切都已就绪——明确的任务、开放的标准、共享的数据以及正确的培训——任何指挥部或单位在使用自主性时面临的最大挑战是“在复杂且不可预测的作战环境中保持持续的高可靠性”。

随着人工智能软件代理因用户请求定义不清而违法的案例不断出现，这个问题——这个软件程序是否理解我的意图？——变得越来越紧迫。

如果 AUTOWARCOM 要为军队指明方向，其领导人必须认识到以前在自主性方面的努力为何成功或失败，并承认人类的信任是无法自动化的。

下一篇报道：美国情报局长被任命为白宫人工智能沙皇