Existential threats join bubble fears as AI mood sobers at Singapore forums

SINGAPORE - Existential and financial risks posed by the artificial intelligence boom were top of mind for investors and policymakers at two conferences in Singapore on Wednesday (Oct 7), highlighting a cautious shift in mood around the AI trade.

Concerns that once centred on frothy valuations have broadened into a wide swathe of fears, including autonomous systems slipping out of human control, AI-designed bioweapons and the possibility that massive spending on the technology may never pay off.

"Anyone who tells you I've got absolute control over my autonomous agent... obviously doesn't know what he's talking about. So loss of control is a real threat," Singaporean Foreign Minister Vivian Balakrishnan said at the Milken Institute Asia Summit.

Last month, the heads of leading US AI labs came together in a rare show of unity to call for slowing the technology's development, warning it could soon improve on its own and slip beyond human control.

Claude maker Anthropic plans to caution potential investors in its ​IPO that advanced AI could pose "catastrophic or existential risks to humanity," Reuters reported last week.

Balakrishnan said on Wednesday the risk of state and non-state actors using the technology to build new weapons, "whether it's in chemistry or biology or in weapons of mass destruction, is very real."

Kiran Mazumdar-Shaw, founder and chairperson of Indian biopharmaceutical firm Biocon, said that while AI was a force for good, bad actors could use it to design new viruses or pathogens.

"The real existential threat is bioterrorism," she said at the Forbes Global CEO Conference, calling for guardrails and governance. "I don't think self-governance is going to work."

Bubble worries linger

Investors' faith in the AI trade has propped up the stock market even as higher energy prices and episodic routs in the bond market have pushed borrowing costs in major economies to multi-decade highs.

But worries over how long the AI rally can last and what it means for global markets have persist.

"If the AI narrative is unwound because of safety issues, regulation or because as we go through 2027, end users don't see sufficient ROI (return on investment), there could be issues," said Rohit Sipahimalani, chief investment officer at Singapore state investor Temasek.

"The biggest risk to the market is the AI narrative. Right now, it seems very strong," Sipahimalani said at the Milken Institute event.

Bridgewater Associates founder Ray Dalio was less sanguine, saying AI was in a "classic bubble" and there was a difference between "how terrific a technology is and how terrific an investment is."

He told the Forbes event that rising rates and changing circumstances had not "yet had a negative effect on other things," but that there was pressure for interest rates to rise further.

Once higher rates began to bite, the bubble would start to burst, Dalio said. "I think we're close to that."

新加坡——周三（10 月 7 日），在新加坡举行的两场会议上，人工智能热潮带来的生存和金融风险成为投资者和政策制定者最为关注的问题，这凸显了人们对人工智能交易态度的谨慎转变。

曾经主要集中在估值过高的担忧已经扩展到一系列广泛的恐惧，包括自主系统脱离人类控制、人工智能设计的生物武器以及对该技术的巨额投入可能永远不会获得回报。

新加坡外交部长维文·巴拉克里希南在米尔肯研究院亚洲峰会上表示：“任何告诉你我对我的自主代理拥有绝对控制权的人……显然都不知道他在说什么。因此，失去控制权是一个真正的威胁。”

上个月，美国顶尖人工智能实验室的负责人罕见地团结起来，呼吁放缓人工智能技术的发展速度，并警告说，人工智能可能很快就会自行改进，并超出人类的控制。

据路透社上周报道，Claude 的制造商 Anthropic 计划在其 IPO 中提醒潜在投资者，先进的人工智能可能会给人类带来“灾难性或生存性风险”。

巴拉克里希南周三表示，国家和非国家行为体利用该技术制造新型武器的风险“无论是在化学、生物学还是大规模杀伤性武器领域，都是非常现实的”。

印度生物制药公司 Biocon 的创始人兼董事长 Kiran Mazumdar-Shaw 表示，虽然人工智能是一股向善的力量，但坏人可能会利用它来设计新的病毒或病原体。

她在福布斯全球CEO大会上表示：“真正的生存威胁是生物恐怖主义”，并呼吁建立保障措施和治理机制。“我认为自治行不通。”

泡沫担忧挥之不去

尽管能源价格上涨和债券市场出现间歇性暴跌，导致主要经济体的借贷成本达到数十年来的最高水平，但投资者对人工智能交易的信心仍然支撑了股市。

但人们仍然担心人工智能热潮能持续多久，以及这对全球市场意味着什么。

新加坡主权财富基金淡马锡首席投资官罗希特·西帕希马拉尼表示：“如果人工智能的发展因安全问题、监管问题，或者因为到 2027 年最终用户看不到足够的投资回报率而受阻，可能会出现问题。”

“市场面临的最大风险是人工智能的叙事。目前来看，这种叙事似乎非常强势，”西帕希马拉尼在米尔肯研究院的活动上表示。

桥水基金创始人雷·达里奥则没那么乐观，他表示人工智能正处于“典型的泡沫”之中，而且“一项技术有多棒和一项投资有多棒之间是有区别的”。

他在福布斯活动上表示，利率上升和环境变化“尚未对其他方面产生负面影响”，但利率面临进一步上升的压力。

达利欧表示，一旦高利率开始产生影响，泡沫就会开始破裂。“我认为我们离那一天已经很近了。”