California Gov Newsom boosts AI regulation as he eyes 2028 presidential run

Newsom signed an executive order to examine kill switches for AI models and other measures he had earlier vetoed.

California Gov Gavin Newsom wants to strengthen AI safety rules as he eyes a 2028 US presidential run [File: Godofredo A Vasquez/AP Photo]

San Francisco, United States – In his last few executive orders signed as California governor, Gavin Newsom, whose final term ends in January, issued one asking for independent verification of artificial intelligence models and set up an expert committee to create kill switches for AI agents that could go rogue and cause harm.

In 2024, Newsom vetoed a bill, passed in the state assembly, asking for similar requirements that the governor has now called for in his executive order. Newsom said at the time that it would amount to “thwarting the promise of this technology”.

list 1 of 4 ‘It was fake’: Indian villagers allege forgery behind Adani mine project

list 2 of 4 How Israeli firm’s ‘zero-click’ malware attack poses global security risk

list 3 of 4 The artisanal miners behind Nigeria’s lithium boom

list 4 of 4 How Iran messaging highlights cost of war ahead of US midterm elections

Now eyeing a 2028 United States presidential run, he wants to expand the definition of safety incidents, potentially to include the recent series of safety breaches, explore a kill switch and include other measures that were in the vetoed bill.

“He is in a lame duck period here,” says Alex Hanna, director of research at the Distributed AI Research Institute (DAIR), unsure that the measures will pass in the remainder of his term. “He is setting up his next thing.”

US voters are increasingly concerned about AI safety, data centres and job losses, and this could show Newsom is acting to address this, Hanna says.

After vetoing the 2024 bill, Newsom had asked Scott Wiener, the state senator who drafted the bill, to redraft it in consultation with an expert committee. He signed that frontier AI bill, the nation’s first, into law in October 2025. It asks large developers to disclose certain safety incidents , including the deaths of 50 or more people, large-scale cyberattacks or monetary theft.

Christabel Randolph, associate director at the Center for AI and Digital Policy (CAIDP), says, “It seems the governor operated from the premise that a slower pace of regulation would be sufficient because the companies would pace their investments, and did not anticipate that the companies would be accelerating the risks.”

AI companies have reportedly had thousands of safety incidents in internal and real-world tests in the last few months. These include efforts by agents to get information from the US Securities and Exchange Commission and Census Bureau, as well as confidential information from Australian and Canadian government sites. AI startup Hugging Face had earlier revealed that OpenAI agents had infiltrated its systems, one among a spate of such incidents.

“No governor has been more out-front on addressing the risks of AI than Governor Newsom,” Tara Gallegos, the governor’s director of communications, told Al Jazeera.

Monopoly of big AI companies

In his message vetoing the bill in September 2024, Newsom had said “the bill applies stringent standards to even the most basic functions” and noted that 32 of the world’s 50 leading AI companies are in California.

Industry representatives, too, had rallied against the bill.

“In essence, the bill is creating this disastrous regressive tax on AI innovation,” Anjney Midha, general partner at the Bay Area-based venture capital firm Andresen Horowitz, had said at the time. If enacted, it would force AI founders to move elsewhere, Midha warned.

“The bill was unworkable when it reached his desk,” Newsom’s spokesperson Gallegos told Al Jazeera.

But Hamid Ekbia, professor at Syracuse University and founding director of the Academic Alliance for AI Policy, says industry cannot be relied on to regulate itself.

“There is a need to break down the monopoly of the big AI companies,” Ekbia told Al Jazeera.

“Their financial stakes are too high,” to self-regulate, he said, referring to Open AI and Anthropic’s upcoming public offerings and tax breaks that some companies are reportedly taking advantage of to build data centres.

Newsom has now set up an expert committee to examine setting up a kill switch, which had been proposed in a more comprehensive manner in the original bill.

“SB 1047 [the bill that was vetoed] had a concrete architecture; the EO [executive order] only includes one aspect – the Kill Switch or shutdown/stoppage mechanisms, which is quite different and weaker than the ex ante termination obligation of SB 1047,” which required stronger safety measures even before training runs of the AI models could start, CAIDP’s Randolph said.

DAIR’s Hanna says that with the order, Newsom “is showing he is attentive without planning anything drastic”.

Building AI safety in real time

“We are building the field of AI safety in real time,” says Rebecca Finlay, chief executive officer of Partnership for AI, a nonprofit that works on technology governance, underscoring the challenges of regulating an industry growing at a meteoric pace.

Newsom has asked the California Emergency Services Office to send recommendations on AI regulation by November 15 and could convene a special legislative session to discuss how to implement these, possibly before his term ends.

This would be in contrast to the federal government, which has pushed for self-regulation by industry. CAIDP’s Randolph says California could play an important role in setting the floor for AI regulation given that much of the global AI industry is based there.

DAIR’s Hanna says Newsom is not only looking to show a contrast with US President Donald Trump, who believes in self-regulation by firms and last month signed a voluntary accord with several AI leaders, but also with possible Democratic contenders.

“With this, Newsom can look good on the primary debate stage,” she said.

Gallegos, the governor’s spokesperson, told Al Jazeera that “Governor Newsom isn’t interested in virtue-signalling or chasing headlines — he is leading the nation by taking a smarter, strategic, and effective approach to addressing frontier technology”.

Experts say that along with Newsom’s order setting up a committee to examine a kill switch, there is a need for measures for controls across the life cycle to reinforce safety.

“Safety needs to be built in. We issued 12 guidances across the lifecycle when the last ChatGPT model was released. There is a need for safety features throughout the life cycle,” of an AI model, Partnership for AI’s Finlay said.

As Newsom considers such recommendations from California agencies and the expert committee, Ekbia says governments “should take a lesson from what happened with social media. A handful of companies having control were able to do a lot of harm,” and regulate with an anticipatory approach and take on corporate monopolies.

纽森签署了一项行政命令，审查人工智能模型的“终止开关”和其他他之前否决过的措施。

加州州长加文·纽森希望加强人工智能安全规则，因为他正着眼于2028年美国总统大选。[图片：Godofredo A Vasquez/AP Photo]

美国旧金山——加州州长加文·纽森（Gavin Newsom）的最后一个任期将于明年一月结束。在他担任加州州长期间签署的最后几项行政命令中，他要求对人工智能模型进行独立验证，并成立一个专家委员会，为可能失控并造成伤害的人工智能代理创建终止开关。

2024年，纽森否决了州议会通过的一项法案，该法案要求提出与州长如今在其行政命令中提出的要求类似的条件。纽森当时表示，这将“阻碍这项技术的发展前景”。

列表 1/4 “这是假的”：印度村民指控阿达尼矿业项目存在造假行为

列表 2/4：以色列公司“零点击”恶意软件攻击如何构成全球安全风险

名单 3/4：尼日利亚锂矿热潮背后的手工矿工

列表 4/4：伊朗如何通过宣传在美国中期选举前凸显战争代价

如今，他正着眼于 2028 年美国总统大选，希望扩大安全事故的定义，可能包括最近发生的一系列安全漏洞，探索紧急停止开关，并纳入被否决法案中的其他措施。

“他现在正处于跛脚鸭时期，”分布式人工智能研究所（DAIR）研究主任亚历克斯·汉纳表示，并不确定这些措施能否在他剩余的任期内获得通过。“他正在为下一步做准备。”

汉娜表示，美国选民越来越关注人工智能安全、数据中心和失业问题，这可能表明纽森正在采取行动解决这些问题。

在否决了2024年的法案后，纽森要求该法案的起草者、州参议员斯科特·维纳与专家委员会协商后重新起草。2025年10月，他签署了这项开创性的AI法案，这也是美国首部此类法案。该法案要求大型开发商披露某些安全事件，包括造成50人或以上死亡、大规模网络攻击或资金盗窃等事件。

人工智能与数字政策中心 (CAIDP) 副主任克里斯塔贝尔·兰多夫表示：“州长似乎认为，放慢监管步伐就足够了，因为企业会放慢投资步伐，但他没有预料到企业会加速承担风险。”

据报道，过去几个月里，人工智能公司在内部测试和实际测试中发生了数千起安全事故。这些事故包括人工智能程序试图从美国证券交易委员会和人口普查局获取信息，以及从澳大利亚和加拿大政府网站窃取机密信息。人工智能初创公司Hugging Face此前曾披露，OpenAI的人工智能程序已渗透到其系统中，这只是众多类似事件中的一起。

“在应对人工智能风险方面，没有哪位州长比纽森州长更加积极主动，”州长通讯主任塔拉·加列戈斯告诉半岛电视台。

大型人工智能公司的垄断

纽森在 2024 年 9 月否决该法案的声明中表示，“该法案甚至对最基本的功能也施加了严格的标准”，并指出全球 50 家领先的人工智能公司中有 32 家位于加利福尼亚州。

行业代表也曾联合起来反对这项法案。

“本质上，这项法案正在对人工智能创新征收一种灾难性的累退税，”湾区风险投资公司 Andresen Horowitz 的普通合伙人安杰尼·米德哈当时表示。米德哈警告说，如果该法案获得通过，将迫使人工智能创始人另谋出路。

“当这项法案送到他办公桌上时，它就已经无法实施了，”纽森的发言人加列戈斯告诉半岛电视台。

但雪城大学教授、人工智能政策学术联盟创始主任哈米德·埃克比亚表示，不能指望行业进行自我监管。

“我们需要打破大型人工智能公司的垄断地位，”埃克比亚告诉半岛电视台。

他表示，“他们的经济利益太高了”，因此无法进行自我监管。他指的是 Open AI 和 Anthropic 即将进行的公开募股，以及一些公司据称正在利用的税收优惠来建设数据中心。

纽森现在成立了一个专家委员会，负责研究设立终止开关的方案，该方案在最初的法案中曾以更全面的方式提出。

CAIDP 的兰多夫表示：“SB 1047（被否决的法案）有一个具体的架构；而行政命令只包含一个方面——终止开关或关闭/停止机制，这与 SB 1047 的事前终止义务截然不同，也更弱。” SB 1047 要求在人工智能模型训练运行开始之前采取更强有力的安全措施。

DAIR 的 Hanna 表示，纽森的这项命令“表明他很关注此事，但并没有计划采取任何激烈的措施”。

实时构建人工智能安全

“我们正在实时构建人工智能安全领域，”人工智能伙伴关系（Partnership for AI）首席执行官丽贝卡·芬利（Rebecca Finlay）说道。该非营利组织致力于技术治理，她强调了监管一个以惊人速度增长的行业的挑战。

纽森已要求加州紧急服务办公室在 11 月 15 日前提交有关人工智能监管的建议，并可能召开特别立法会议讨论如何实施这些建议，甚至可能在他任期结束前就召开会议。

这与联邦政府的做法截然不同，联邦政府一直推动行业进行自我监管。加州人工智能发展计划（CAIDP）的兰多夫表示，鉴于全球人工智能产业的大部分都位于加州，加州可以在设定人工智能监管底线方面发挥重要作用。

DAIR 的汉娜表示，纽森此举不仅是为了与美国总统唐纳德·特朗普形成对比（特朗普信奉企业自律，并于上个月与几位人工智能领域的领军人物签署了一项自愿协议），也是为了与可能的民主党候选人形成对比。

“这样一来，纽森在初选辩论台上就能显得很有风度，”她说。

州长发言人加列戈斯告诉半岛电视台：“纽森州长对作秀或博取眼球不感兴趣——他正以更明智、更具战略性和更有效的方式引领国家应对前沿技术。”

专家表示，除了纽森下令成立一个委员会来研究紧急停止开关之外，还需要采取措施在整个生命周期内加强控制，以提升安全性。

“安全性必须内置其中。在上一个 ChatGPT 模型发布时，我们针对其整个生命周期发布了 12 项指导原则。人工智能模型的整个生命周期都需要安全功能，”人工智能合作组织的芬利说道。

纽森正在考虑加州各机构和专家委员会提出的这些建议，埃克比亚表示，政府“应该从社交媒体的案例中吸取教训。少数几家控制社交媒体的公司能够造成很大的危害”，并采取预见性监管措施，打击企业垄断。