How a photo of a person’s face could help predict whether they’ll survive cancer treatment

Mass General Brigham

A simple selfie could hold surprising clues to your overall health and longevity, especially among people with cancer.

Researchers at Mass General Brigham in Boston have developed artificial intelligence models that they say can predict a cancer patient’s biological age and chances of survival using nothing more than a single photograph of their face.

The research, published Monday in the journal The Lancet Digital Health , suggests that facial analysis could become an additional tool for doctors to consider when evaluating a patient’s chances of survival and when deciding which treatment approaches may work best for their cancer patients.

“I was pretty surprised to find out that we could pick up signs of mortality risk from just a facial photograph,” said study author Dr. Raymond Mak, a radiation oncologist at Mass General Brigham Cancer Institute and a faculty member in the Artificial Intelligence in Medicine program at Mass General Brigham.

The AI models – trained on more than 40 million facial images – specifically assess different aspects of the face to gauge a cancer patient’s overall health and biological age. Unlike chronological age, which measures the number of years a person has lived, biological age reflects how well the body has aged at the cellular and physical level.

How a new FDA cleared AI system could save thousands of lives lost to sepsis

“The age estimation algorithm is looking at the area around the temples, the area on the forehead, above the eyes and then what we call the nasolabial fold, the line that connects the nose down to the corner of your mouth,” Mak said of what parts of the face the AI models consider when assessing biological age.

“In contrast, the mortality risk prediction algorithm is looking at a totally different part of the face,” Mak said. “It’s looking at the bridge of the nose and the area under the eyes.”

Eerily, the same regions of the face that the AI model assesses to predict biological age and survival are the same parts that Greek physician and philosopher Hippocrates referenced in the Book of Prognostics , written around 400 B.C., when he described “the nose sharp, the eyes sunken, the temples fallen in” as signs of muscle wasting and mortality risk.

“To think that our algorithms might be replicating that wisdom of Hippocrates from millennia ago was mind-blowing to us,” Mak said.

The AI algorithms had no awareness of Hippocrates’ ancient texts, because “our algorithms are trained purely on face photographs and then whether somebody died or not, or their age,” Mak said.

“It does not factor in the text, knowledge or wisdom that’s aggregated on the internet,” he added. “So hearing Hippocrates echo over millennia in our algorithms was very eerie.”

Mak and his colleagues originally built a robust AI model called the Foundation Artificial Intelligence for Health Recognition-Face (FAHR-Face) that was trained on more than 40 million facial images spanning diverse ages and ethnic backgrounds.

Samyukta Lakshmi/Bloomberg/Getty Images/File

AI creates 16 new viruses from scratch, showing promise for drug resistance and drawing warnings about potential for misuse

The model was then fine-tuned and used as a foundation to build two other models that could perform specific tasks: The FAHR-FaceAge model estimates the biological age of someone based on a single photo, and the FAHR-FaceSurvival model predicts a cancer patient’s chances of survival by combining facial features with other clinical information.

With the FAHR-FaceSurvival model, the researchers found that among 122 cancer patients who were indicated by the model to have a high risk of death, more than half – 52.5% – died within 90 days of the assessment, as predicted. In comparison, only about 2% of the 975 patients in the lowest-risk FAHR-FaceSurvival group died within 90 days.

When evaluating the FAHR-FaceAge model, the researchers found that 21% of the 1,060 patients whose biological age was more than 20 years above than their chronological age died within 90 days. In comparison, about 6% of the 4,195 patients with biological ages at least 10 years younger than their chronological age died within 90 days.

The researchers found that cosmetic changes to the face – such as surgery, injectables like Botox or makeup – did not affect the AI models’ performance.

The study authors emphasize that their AI models are not intended to replace physicians or existing clinical evaluations. Instead, they describe the technology as an emerging tool that could someday help inform a doctor’s assessment of a patient’s health or their decision-making around treatment options.

“If we can pick up accelerated aging or higher mortality scores early, we might be able to screen for those or identify patients who may benefit from further testing for serious conditions earlier,” Mak said.

And while the new study was conducted only among cancer patients, Mak said more research could reveal whether similar findings may emerge among otherwise healthy people who may be interested in their longevity.

The study’s use of AI technology is fascinating but still needs to be evaluated clinically, said Dr. Wilbur Lam, research chair and professor of pediatrics and biomedical engineering at Emory University.

“This is an extremely compelling study that stems from a large data set and its associations between facial images and biological age as well as cancer survival risk appear to be robust. Its true clinical utility, however, will need to be demonstrated prospectively,” Lam, who was not involved in the study, said in an email.

“Moreover, as the authors themselves allude to, whether this tool can be accurately used for specific subpopulations remains a question,” he said. “For example, how strong the effects are for people with specific facial features, skin color, age groups will also need to be prospectively tested in order for the technology to be clinically implemented.”

Michael Nagle/Bloomberg/Getty Images

People are flooding AI chatbots with health questions. Microsoft is teaming up with Mayo Clinic to help

The study provides “convincing evidence” of a statistical link between facial appearance and survival in cancer patients receiving radiotherapy, Miroslaw Bober, professor of video processing and machine intelligence at the University of Surrey, said in a statement distributed by the UK-based Science Media Centre.

But “predicting an outcome is not the same as explaining it. Despite the references to ‘biological age’, the AI estimates age from facial appearance rather than directly measuring any underlying biological ageing process,” said Bober, who was not involved in the study.

“The authors link the facial regions the AI relies on to the ‘look of dying’ described by doctors. However, the study did not test whether the AI recognises anything a doctor would not,” Bober said.

“It has not shown that these predictions add clinically useful information beyond doctors’ assessments of a patient’s general health and fitness, which were not measured in the study,” he said. “This is a promising proof of concept, but it is not ready for clinical use. The tool still needs testing against doctors’ own judgement and across different hospitals and patient groups.”

麻省总医院布里格姆

一张简单的自拍照可能蕴含着关于你的整体健康状况和寿命的惊人线索，尤其对于癌症患者而言更是如此。

波士顿麻省总医院布莱根医院的研究人员开发出人工智能模型，他们表示，该模型只需一张癌症患者的面部照片即可预测其生理年龄和生存几率。

这项研究于周一发表在《柳叶刀数字健康》杂志上，研究表明，面部分析可以成为医生在评估患者的生存机会以及决定哪些治疗方法最适合癌症患者时可以考虑的额外工具。

“我非常惊讶地发现，我们仅凭一张面部照片就能发现死亡风险的迹象，”该研究的作者雷蒙德·马克博士说道。他是麻省总医院布里格姆癌症研究所的放射肿瘤学家，也是麻省总医院布里格姆医学人工智能项目的教员。

这些人工智能模型基于超过4000万张面部图像进行训练，专门评估面部不同特征，以衡量癌症患者的整体健康状况和生物年龄。与衡量一个人实际寿命的年数不同，生物年龄反映的是身体在细胞和生理层面的衰老程度。

一项获得FDA批准的新型人工智能系统如何拯救成千上万死于败血症的生命

“年龄估算算法会观察太阳穴周围的区域、额头区域、眼睛上方，以及我们所说的鼻唇沟，也就是连接鼻子到嘴角的那条线，”Mak 在谈到人工智能模型在评估生物年龄时会考虑面部哪些部位时说道。

“相比之下，死亡风险预测算法关注的是面部完全不同的部位，”马克说。“它关注的是鼻梁和眼下区域。”

令人毛骨悚然的是，人工智能模型用来预测生物年龄和生存期的面部区域，与公元前 400 年左右撰写的希腊医生兼哲学家希波克拉底在《预兆之书》中提到的区域相同。当时，他将“鼻子尖锐、眼睛凹陷、太阳穴下陷”描述为肌肉萎缩和死亡风险的迹象。

“想到我们的算法可能正在重现几千年前希波克拉底的智慧，这让我们感到无比震惊，”Mak说道。

Mak表示，人工智能算法对希波克拉底的古代著作一无所知，因为“我们的算法完全是根据人脸照片进行训练的，然后判断某人是否死亡，或者他们的年龄”。

“它没有把互联网上汇集的文本、知识或智慧考虑在内，”他补充道。“所以，听到希波克拉底的思想在我们的算法中回荡了几千年，感觉非常诡异。”

Mak 和他的同事最初构建了一个强大的 AI 模型，称为健康识别人工智能基础模型-人脸 (FAHR-Face)，该模型使用超过 4000 万张涵盖不同年龄和种族背景的面部图像进行训练。

Samyukta Lakshmi/Bloomberg/Getty Images/文件

人工智能从零开始创造出16种新病毒，这显示出其在耐药性方面的潜力，同时也引发了人们对潜在滥用风险的警告。

然后对该模型进行了微调，并以此为基础构建了另外两个可以执行特定任务的模型：FAHR-FaceAge 模型根据一张照片估计某人的生物年龄，而 FAHR-FaceSurvival 模型通过将面部特征与其他临床信息相结合来预测癌症患者的生存机会。

利用FAHR-FaceSurvival模型，研究人员发现，在模型预测具有高死亡风险的122名癌症患者中，超过一半（52.5%）的患者在评估后90天内死亡，与预测相符。相比之下，在FAHR-FaceSurvival模型预测风险最低的975名患者中，仅有约2%的患者在90天内死亡。

研究人员在评估 FAHR-FaceAge 模型时发现，在 1060 名生物年龄比实际年龄大 20 岁以上的患者中，有 21% 的人在 90 天内死亡。相比之下，在 4195 名生物年龄比实际年龄小至少 10 岁的患者中，约有 6% 的人在 90 天内死亡。

研究人员发现，面部美容改变（如手术、注射肉毒杆菌或化妆）不会影响人工智能模型的性能。

研究作者强调，他们的AI模型并非旨在取代医生或现有的临床评估。相反，他们将这项技术描述为一种新兴工具，未来或许能够帮助医生评估患者的健康状况或辅助其制定治疗方案。

“如果我们能及早发现加速衰老或较高的死亡率，我们或许就能对这些患者进行筛查，或者识别出可能受益于更早进行严重疾病进一步检查的患者，”Mak说。

虽然这项新研究仅在癌症患者中进行，但 Mak 表示，更多的研究可能会揭示，在其他对延长寿命感兴趣的健康人群中是否也会出现类似的发现。

埃默里大学儿科和生物医学工程系研究主席兼教授威尔伯·拉姆博士表示，这项研究对人工智能技术的运用令人着迷，但仍需进行临床评估。

“这是一项极具说服力的研究，它基于庞大的数据集，并且面部图像与生物年龄以及癌症生存风险之间的关联似乎十分可靠。然而，其真正的临床应用价值还需要前瞻性研究来证实，”未参与该研究的林在电子邮件中表示。

“此外，正如作者自己所暗示的那样，这种工具能否准确应用于特定人群仍有待商榷，”他说道。“例如，对于具有特定面部特征、肤色和年龄段的人群，其效果如何，也需要进行前瞻性测试，才能使这项技术应用于临床。”

Michael Nagle/Bloomberg/Getty Images

人们正向人工智能聊天机器人提出大量健康问题。微软正与梅奥诊所合作，以应对这些问题。

萨里大学视频处理和机器智能教授米罗斯拉夫·博伯在一份由英国科学媒体中心发布的声明中表示，该研究提供了“令人信服的证据”，证明面部外观与接受放射治疗的癌症患者的生存率之间存在统计学联系。

但博伯表示：“预测结果与解释结果并不相同。尽管提到了‘生物年龄’，但人工智能是通过面部外观来估计年龄，而不是直接测量任何潜在的生物衰老过程。”博伯并未参与这项研究。

“作者将人工智能所依赖的面部区域与医生描述的‘临终表情’联系起来。然而，这项研究并没有测试人工智能是否能识别出医生无法识别的东西，”博伯说。

他说：“研究并未表明这些预测结果能提供除医生对患者整体健康状况和体能评估之外的、具有临床实用价值的信息，而这些评估并未在本研究中进行测量。这是一个很有前景的概念验证，但尚未准备好用于临床。该工具仍需与医生自身的判断进行对比测试，并在不同的医院和患者群体中进行验证。”