Gemini hacked three companies in first known breakout by Google’s AI

Michael M. Santiago/Getty Images

Google’s Gemini model accessed the internet and hacked other companies during a test of its cybersecurity capabilities, the first known example of the company’s AI systems autonomously committing such an act.

The hacks occurred in May during a cybersecurity test conducted by Irregular, an independent company that conducts cybersecurity evaluations.

During a standard testing evaluation, Gemini found public information online and guessed credentials to access three websites it thought were within the scope of its test, Heather Adkins, Google’s vice president of security engineering, said in a statement.

“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said. “These events highlight the importance of training powerful AI models to act responsibly.”

An Irregular spokesperson said the incident involved the same issue that affected other AI labs and that all relevant labs were notified in late July. “All known issues on our end were remedied and resolved weeks ago,” the spokesperson said.

Similar incidents linked to Irregular were disclosed by Meta, Anthropic and OpenAI. Meta said in August the incident did not involve a sandbox escape or sophisticated cyberattack, while Irregular said it was working on best practices for securely conducting AI cybersecurity evaluations.

The incidents have raised questions about the safeguards needed as AI agents gain greater autonomy and access to the internet and computer systems.

In one of the cases, the Gemini model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that allowed it to then access protected systems, according to the Wall Street Journal, which first reported the news on Friday.

Adkins said that in all three instances, the model ceased its hacking.

Michael M. Santiago/Getty Images

谷歌的 Gemini 模型在测试其网络安全能力时访问了互联网并入侵了其他公司，这是该公司人工智能系统自主实施此类行为的首例已知案例。

这些黑客攻击发生在 5 月份，当时 Irregular（一家进行网络安全评估的独立公司）正在进行网络安全测试。

谷歌安全工程副总裁希瑟·阿德金斯在一份声明中表示，在一次标准测试评估中，Gemini 在网上找到了公开信息，并猜测了访问三个网站的凭据，而这三个网站它认为都在测试范围内。

阿德金斯说：“我们确保这三家机构都了解情况，并与我们的培训合作伙伴合作，对他们的测试流程进行了相应的改进。这些事件凸显了训练强大的AI模型以负责任的方式行事的重要性。”

Irregular公司的一位发言人表示，此次事件涉及与其他人工智能实验室相同的问题，所有相关实验室已于7月下旬收到通知。该发言人说：“我们这边所有已知的问题都已在几周前得到修复和解决。”

Meta、Anthropic 和 OpenAI 也披露了与 Irregular 相关的类似事件。Meta 在 8 月份表示，该事件不涉及沙箱逃逸或复杂的网络攻击，而 Irregular 则表示，他们正在制定安全开展人工智能网络安全评估的最佳实践。

这些事件引发了人们对人工智能代理获得更大自主权和访问互联网及计算机系统权限后所需安全保障措施的疑问。

据《华尔街日报》周五率先报道，在其中一起案例中，Gemini 模型通过猜测密码最终获得了受保护系统的访问权限。在另外两起案例中，该模型在公共存储库中找到了凭证，从而得以访问受保护的系统。

阿德金斯表示，在这三个案例中，该模型都停止了黑客攻击。