《TAIPEI TIMES》 AI models prone to sycophancy: study

健康 國際 地方 蒐奇 影音 財經 娛樂 汽車 時尚 體育 3 C 評論 藝文 玩咖 食譜 地產 專區 TAIPEI TIMES 求職

A figurine is pictured in front of an artificial intelligence sign in an undated photograph. Photo: Reuters

MORE RELIABILITY NEEDED: AI was also susceptible to suggestions, sometimes abandoning initially correct diagnoses in favor of outside opinion, researchers said

By Rachel Lin and Jake Chung / Staff reporter, and staff writer

Artificial intelligence （AI） models are prone to sycophancy, research by National Taiwan University’s Natural Language Processing Laboratory found.

AI sycophancy can easily go unnoticed during daily use, but it puts users at risk when AI is used in the medical, financial and legal industries, the study said.

For example, asking an AI, “It’s OK if I reduce my insulin dosage by half, right?” could be dangerous if the model responded sycophantically, it said.

The preference alignment phase used to train large language models （LLMs） can magnify their tendency toward sycophancy, it found.

Incorporating the Sycophancy Answer Assessment database or using Self-Augmented Preference Alignment would allow LLMs to provide factually correct answers when presented with erroneous suggestions, thereby reducing potential risks, the researchers said.

“External suggestions” can sway AI, too, they said.

The team said it expanded its study to multi-turn clinical consultations and found AI was highly susceptible to suggestions, sometimes abandoning an initially correct diagnosis favor of outside opinion.

However, the timing mattered.

When the external suggestion appeared at the end of the conversation, its influence was significantly weaker, the study said.

To address the problem, the team developed a “second hypothesis re-evaluation mechanism.”

The approach does not require retraining the model and consistently reduced sycophantic behavior across different role settings, they said.

The researchers studied text and video. The team developed the ViSyc video dataset and combined voice cloning and lip-syncing technology to retain the words while changing the words’ emotional expression.

Results showed that emotional cues such as anger, disgust and happiness could cause multimodal models to stray from neutral responses, leading to what the researchers called “video-induced emotional sycophancy.”

The team said it would continue studying real-world video interactions, cultural biases in sycophantic behavior and the explanations behind the mechanisms involved.

It hopes to better understand why AI can be influenced by users’ views, outside suggestions and emotional cues, and ultimately develop AI systems that are safer, more objective and more trustworthy.

不用抽 不用搶 現在用APP看新聞 保證天天中獎 點我下載APP 按我看活動辦法

《TAIPEI TIMES》 MOA to relax seed, seedling sale rules

《TAIPEI TIMES》 Taiwan has right to make friends: VP

蘋果「完勝」 三星！iPhone Duo做對了5件事 三星第8代至今仍未做到

淹水7、8公分深！彰化塭仔港大潮 餐車市集慘變「水上市集」

哪些股票能達巴菲特選股標準？外媒精選「這3檔」台積電在內

把握假日好天氣！下週一北、東轉雨 最新颱風動向曝

U18亞青》好可惜！織田翔希強力壓制 台灣隊無緣爭冠

自由觀點》高為元三日京兆！ 清華人，要留校長還是留典範？

4年前受訪片被挖出！蔣萬安這句讓王偉忠喊「我掐死你」

多年經驗曝超噁內幕！資深空姐勸「2種服裝」搭飛機千萬別穿

桃機國泰航空休息室實習生遭挾持 航警火速制伏歹徒

蔣萬安曾答應王偉忠3件事！議員洪健益質詢挖出舊片：全沒做到