AI and the end of humanity? OK, ‘doomer’ says one Trump official

Samuel Boivin/NurPhoto/Getty Images/File

A version of this story appeared in CNN’s What Matters newsletter. To get it in your inbox, sign up for free here .

The latest warnings about AI seem intended to boggle minds and shock everyone into action.

Anthropic CEO Dario Amodei, a leader in the AI race, on Saturday called for a conscious slowdown in AI development in an essay posted to his website. AI is developing faster and faster and could outrun humans’ ability to manage it, he wrote.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” Amodei wrote.

The call for action comes just days after a former engineer at Amodei’s company went even further, saying AI could bring about the end of humanity.

“The people building AI earnestly believe that it could kill us all by the end of the decade,” said Jacob Coxon , who had worked at both OpenAI and Anthropic before he posted his viral warning on social media.

Nobody knows how exactly AI would end humanity, as Coxon explained to CNN’s Anderson Cooper.

Former AI researcher Jacob Coxon tells CNN’s Anderson Cooper why he quit Anthropic over fears that rapidly advancing AI could become impossible to control.

The argument is that AI is getting smarter all the time and will soon make itself smarter on its own, be able to outwit humans and unleash … something. Hack nuclear weapons? Let loose a devastating virus? Plunge the world into darkness? Create a lame society in which unfulfilled people can’t find work? Take your pick.

It should be comforting, maybe, that while the Pentagon is relying on artificial intelligence, Secretary of Defense Pete Hegseth has said, “ AI is not making lethal decisions .” Beyond that, the law does not place explicit limits on where AI can be used in the so-called kill chain.

Coxon’s message was amplified by public interest groups and lawmakers looking to heed the advice of the AI companies that want the government to impose a framework and limits on how AI is developed.

Companies such as Anthropic and OpenAI have publicly called for the government to regulate them so it can slow down their current race for AI domination, which coincides with their race to initial public offerings .

What we’re seeing is “a very small and powerful group of companies, linked arms together, running toward what they say is a cliff they cannot seem to stop themselves from running off,” the tech journalist and CNN contributor Jacob Ward told CNN’s Kasie Hunt on Thursday.

He argued that calls from these people who are in the process of making a lot of money should not be ignored.

“They are saying, ‘Hey, we’re building this thing, and it’s exciting, but it might not just cost you your job, but your life.’ I think it is important that we start paying attention,” Ward said.

Among those calling for government action is Sen. Bernie Sanders, who takes the warnings seriously.

“Nearly every day, there is a frightening new story about how Big Tech companies are losing control of the technology they are developing, with potentially cataclysmic results,” Sanders said in announcing proposed legislation to ban artificial superintelligence. “It is irresponsible for society to allow them to move forward and make these products even more advanced.”

Sanders also talked about his proposal with CNN’s Kaitlan Collins .

Kayla Bartkowski/Getty Images

David Sacks, a tech investor who is also co-chair of President Trump’s Council of Advisors on Science and Technology, has pushed in the other direction , toward an open market-based approach to AI.

Appearing on Fox News Thursday, Sacks questioned Coxon’s motivations and seeded the idea that his warning was part of a conspiracy by “doomers,” maybe financed by China, to thwart AI innovation in the US.

On Saturday, President Donald Trump echoed that sentiment, saying during a news briefing in Ireland that people worried about the ultimate impacts of AI “are bringing up things that won’t happen.”

Asked whether the industry needs to be regulated, Trump said repeated his earlier statement about competing with China.

“We’re leading China in AI,” he said. “We’re the most sophisticated country in the world, and frankly, I want to keep it that way, because whoever wins AI wins. And we can put guard rails.”

Sacks is among those who argued American companies must continue to innovate rather than allow China to achieve AI supremacy.

“They don’t have this giant influence operation of these doomer groups in China,” Sacks told Fox News’ Bret Baier. “What we have here in the US, though, is we have this huge media campaign against AI and it is turning the public against AI.”

Sacks added of China: “They would love for us to stop AI development over here.”

In a video from his “All-In” podcast, which Sacks posted to his account on X , he argued the AI companies want regulation so that they can effectively control AI. He called it a “frontier market duopoly.”

Government officials aren’t going to have any idea how to regulate AI, he argued.

“So they’re going to turn to the leading labs to say, ‘Hey, how should we regulate it?’ And of course, Anthropic is going to have all the answers on that,” he said, suggesting warnings of AI run amok are part of a play for market domination.

Rather than push the idea that AI has the potential to end humanity, Sacks sees a conspiracy to place power over AI in a few select hands. That, he said in another X post, would be the true danger.

“Keeping innovation decentralized and accessible is the key to avoiding an unsafe and dystopian future where advanced AI capabilities are centralized and controlled by only a few hands,” he said .

That kind of mindset is in line with Trump’s efforts to stop states like California from imposing their own AI regulations in the absence of movement from Washington.

When efforts to ban state regulations stalled in Congress last year, Trump signed an executive action trying to do the same thing.

Tilly Norwood has been billed as the first AI "actor.” During Tilly’s first on-camera “interview,” CNN’s Elizabeth Wagmeister takes a look at whether the controversial creation can actually act. Watch the full report on CNN.com .

The Trump administration has so far viewed AI regulation more as part of a race with China over the technology than as an effort to protect humanity, as CNN’s Hadas Gold explained in her recent story, ‘It feels a little like early Covid’: The messy scramble to regulate AI.

The administration blocked export of some Nvidia chips to China, for instance, or temporarily delayed release of products to assess their impact on national security.

To those who see existential danger in AI, all of this may echo the nuclear arms race of the Cold War, when countries ultimately set aside differences to impose a series of treaties in the 1960s and beyond for Nuclear Non-proliferation, test bans, strategic arms limitations, and more.

But it’s hard to imagine the Trump administration — or China — getting involved in any worldwide treaty effort, although Europe has tried to go the route of a treaty for AI regulations.

The threat posed by nuclear weapons still animates US foreign policy and is a main reason cited by Trump for his war on Iran.

While the most dire warnings about AI have to do with the end of humanity, it is most potent as an issue motivating voters worried about AI encroaching in their backyards.

Americans are unnerved at unsightly data centers that power the hive mind of AI and which tech giants would have dot the landscape, impacting water quality, driving up energy bills, and swallowing up farmland.

Andrew Caballero-Reynolds/AFP/Getty Images/File

Scientists have found an alarming environmental impact of vast data centers

The populist, pitchfork-style opposition to the massive construction projects has been the political story of the summer, crossing party lines and uniting people on the left and right, even as it confuses policymakers like Trump , who see dollar signs.

American voters cannot typically be accused of looking long-term.

They express concern about climate change, but seem to vote based on gas prices. They shudder at the $40 trillion national debt, but often vote for lawmakers who cut taxes and increase spending. And with AI, they appear to be far more motivated by concerns about data centers rather than the potential existential threat.

But it’s also possible that distaste for data centers reflects their role as the most obvious physical manifestation of generative AI. Tech companies promise the technology will change the world, and some experts worry it may end humanity — but many ordinary Americans just find it clunky and off-putting, and they didn’t ask for it to invade so many areas of their lives.

Samuel Boivin/NurPhoto/Getty Images/文件

本文的另一个版本曾刊登在 CNN 的“要闻速递”（What Matters）新闻简报中。如需订阅，请点击此处免费注册。

最近关于人工智能的警告似乎意在令人震惊，并促使每个人采取行动。

人工智能领域的领军企业之一、人智公司首席执行官达里奥·阿莫迪（Dario Amodei）周六在其网站上发表文章，呼吁有意识地放缓人工智能的研发速度。他写道，人工智能的发展速度越来越快，可能会超出人类的掌控能力。

“如果不加以控制，它可能会超出我们理解和控制这些系统的能力，因此必须非常谨慎地对待，甚至可能根本不要对待，”阿莫迪写道。

就在几天前，阿莫迪所在公司的一名前工程师甚至发表了更激进的言论，称人工智能可能会导致人类的终结。随后，有人呼吁采取行动。

“人工智能的开发者们真心相信，到本十年末，人工智能可能会毁灭我们所有人，”雅各布·考克森说道。他曾在 OpenAI 和 Anthropic 工作过，之后在社交媒体上发布了他广为流传的警告。

正如考克森向 CNN 的安德森·库珀解释的那样，没有人知道人工智能究竟会如何终结人类。

前人工智能研究员雅各布·考克森向 CNN 的安德森·库珀讲述了他为何因担心快速发展的人工智能可能变得无法控制而离开 Anthropic 公司。

有人认为，人工智能正在不断变得更聪明，很快就会自主地变得更加聪明，能够智胜人类，并释放……某种力量。入侵核武器？释放毁灭性病毒？让世界陷入黑暗？创造一个死气沉沉的社会，让那些不满足于现状的人找不到工作？任君选择。

或许令人感到欣慰的是，尽管五角大楼依赖人工智能，但国防部长皮特·赫格塞斯曾表示，“人工智能不会做出致命的决定”。除此之外，法律并未明确限制人工智能在所谓“杀伤链”中的应用范围。

科克森的言论得到了公共利益团体和立法者的响应，他们希望听取人工智能公司的建议，这些公司希望政府对人工智能的发展方式施加框架和限制。

Anthropic 和 OpenAI 等公司公开呼吁政府对其进行监管，以减缓它们目前在人工智能领域的主导地位竞赛，而这场竞赛恰逢它们竞相进行首次公开​​募股。

科技记者兼 CNN 撰稿人雅各布·沃德 (Jacob Ward) 周四告诉 CNN 的凯西·亨特 (Kasie Hunt)：“我们看到的是一小群实力强大的公司，它们手挽手地奔向他们所说的悬崖，而他们似乎无法阻止自己跳下去。”

他认为，来自这些正在赚很多钱的人的电话不应该被忽视。

沃德说：“他们说，‘嘿，我们正在建设这个项目，这很令人兴奋，但这不仅可能让你丢掉工作，还可能让你丢掉性命。’我认为我们应该开始重视这个问题了。”

呼吁政府采取行动的人士中包括参议员伯尼·桑德斯，他非常重视这些警告。

桑德斯在宣布拟立法禁止人工智能超级智能时表示：“几乎每天都有令人担忧的新故事，讲述大型科技公司如何失去对其所开发技术的控制，这可能会造成灾难性后果。允许他们继续发展并使这些产品更加先进，对社会来说是不负责任的。”

桑德斯还与 CNN 的凯特兰·柯林斯谈到了他的提案。

Kayla Bartkowski/Getty Images

科技投资者大卫·萨克斯同时也是特朗普总统科技顾问委员会的联合主席，他则朝着相反的方向努力，推动人工智能采用开放的市场化方法。

萨克斯周四出现在福克斯新闻频道，质疑考克森的动机，并散布一种观点，认为他的警告是“末日论者”阴谋的一部分，可能是由中国资助的，目的是阻止美国的人工智能创新。

周六，唐纳德·特朗普总统也表达了类似的观点，他在爱尔兰举行的新闻发布会上说，那些担心人工智能最终影响的人“提出的事情不会发生”。

当被问及该行业是否需要监管时，特朗普重申了他之前关于与中国竞争的说法。

他说：“我们在人工智能领域领先于中国。我们是世界上最先进的国家，坦白说，我希望保持这种状态，因为谁在人工智能领域胜出，谁就赢。而且我们可以设置一些保障措施。”

萨克斯等人认为，美国公司必须继续创新，而不是让中国在人工智能领域取得主导地位。

萨克斯告诉福克斯新闻的布雷特·拜尔：“他们在中国没有这种庞大的末日论调团体影响力运作。但在美国，我们却面临着一场声势浩大的反人工智能媒体运动，这场运动正在让公众对人工智能产生抵触情绪。”

萨克斯补充道，关于中国：“他们非常希望我们停止在这里进行人工智能开发。”

在萨克斯发布到其X账号上的“All-In”播客视频中，他认为人工智能公司希望受到监管，以便有效控制人工智能。他称之为“前沿市场双寡头垄断”。

他认为，政府官员根本不知道如何监管人工智能。

“所以他们会求助于顶尖实验室，问：‘嘿，我们应该如何监管它？’当然，Anthropic 公司肯定会给出所有答案，”他说道，暗示对人工智能失控的警告是争夺市场主导地位的策略之一。

萨克斯并不认为人工智能有可能毁灭人类，而是认为存在一种阴谋，旨在将人工智能的控制权集中在少数人手中。他在另一篇X论坛帖子中指出，这才是真正的危险。

他说：“保持创新分散化和可及性是避免出现不安全和反乌托邦未来的关键，在这种未来里，先进的人工智能能力被集中起来，并由少数人控制。”

这种心态与特朗普阻止加州等州在华盛顿方面没有采取行动的情况下实施自己的人工智能法规的努力是一致的。

去年，当禁止各州制定相关法规的努力在国会受阻时，特朗普签署了一项行政命令，试图达到同样的目的。

蒂莉·诺伍德被誉为首位人工智能“演员”。在蒂莉的首次镜头前“采访”中，CNN记者伊丽莎白·瓦格迈斯特将探究这位备受争议的人工智能是否真的具备表演能力。请访问CNN.com观看完整报道。

正如 CNN 记者 Hadas Gold 在她最近的报道《“感觉有点像新冠疫情早期”：混乱的人工智能监管之争》中所解释的那样，特朗普政府迄今为止更多地将人工智能监管视为与中国在该技术领域展开的竞赛的一部分，而不是保护人类的努力。

例如，政府曾阻止英伟达部分芯片出口到中国，或暂时推迟产品发布，以评估其对国家安全的影响。

对于那些认为人工智能存在生存危险的人来说，这一切可能让人联想到冷战时期的核军备竞赛，当时各国最终搁置分歧，在 20 世纪 60 年代及以后签订了一系列条约，包括核不扩散条约、禁止核试验条约、限制战略武器条约等等。

但很难想象特朗普政府——或者中国——会参与任何全球条约的努力，尽管欧洲曾试图通过条约的方式来制定人工智能法规。

核武器构成的威胁仍然是美国外交政策的核心，也是特朗普发动对伊朗战争的主要原因。

虽然关于人工智能最可怕的警告与人类的终结有关，但它最能激发选民的投票热情，因为他们担心人工智能会侵蚀他们的家园。

美国人对那些为人工智能提供集体智慧的丑陋数据中心感到不安，这些数据中心本应遍布各地，却因为科技巨头的建设而影响水质、推高能源账单并侵占农田。

Andrew Caballero-Reynolds/法新社/盖蒂图片社/档案照片

科学家发现，庞大的数据中心对环境造成了令人担忧的影响。

民众对大规模建设项目的民粹主义式的、如同用干草叉般的反对，成为了今年夏天的政治焦点，它超越了党派界限，团结了左翼和右翼人士，尽管这让像特朗普这样只看到金钱的政策制定者感到困惑。

通常来说，不能指责美国选民目光短浅。

他们表达了对气候变化的担忧，但投票时似乎却受油价影响。他们对高达40万亿美元的国债感到震惊，但往往投票支持那些削减税收、增加开支的立法者。而在人工智能领域，他们似乎更关注数据中心，而非潜在的生存威胁。

但对数据中心的反感也可能反映了它们作为生成式人工智能最明显的物理体现的角色。科技公司承诺这项技术将改变世界，一些专家甚至担心它可能会毁灭人类——但许多普通美国人只是觉得它笨重且令人反感，而且他们也并不希望它侵入生活的方方面面。