News Release

AI may predict response to immune checkpoint blockade in patients with metastatic melanoma

Peer-Reviewed Publication

American Association for Cancer Research

Bottom Line: A computational method that combines clinicodemographic variables with deep learning of pre-treatment histology images could predict response to immune checkpoint blockade among patients with advanced melanoma.

Journal in Which the Study was Published: Clinical Cancer Research, a journal of the American Association for Cancer Research

Authors: Corresponding study author Iman Osman, MD, medical oncologist in the Departments of Dermatology and Medicine (Oncology) at New York University (NYU) Grossman School of Medicine and director of the Interdisciplinary Melanoma Program at NYU Langone's Perlmutter Cancer Center; and corresponding study author Aristotelis Tsirigos, PhD, professor in the Institute for Computational Medicine at NYU Grossman School of Medicine and member of NYU Langone's Perlmutter Cancer Center

Background: "While immune checkpoint inhibitors have profoundly changed the treatment landscape in melanoma, many tumors do not respond to treatment, and many patients experience treatment-related toxicity," said Osman. "An unmet need is the ability to accurately predict which tumors will respond to which therapy. This would enable personalized treatment strategies that maximize the potential for clinical benefit and minimize exposure to unnecessary toxicity."

"Several recent attempts to predict immunotherapy responses do so with robust accuracy but use technologies, such as RNA sequencing, that are not readily generalizable to the clinical setting," said Tsirigos. "Our approach shows that responses can be predicted using standard-of-care clinical information such as pre-treatment histology images and other clinical variables."

How the Study was Conducted: The researchers used data from a training cohort of 121 patients with metastatic melanoma who received immune checkpoint blockade treatment between 2004 and 2018. All patients were treated with first-line anti-CTLA-4 therapy, anti-PD-1 therapy, or a combination of both, and clinical outcomes were recorded as progression of disease or response, which included complete or partial responses (patients with stable disease were excluded for this proof-of-principle study). The researchers used computer algorithms called deep convolutional neural networks (DCCN) to analyze digital images of metastatic tumors of melanoma and identify patterns associated with treatment response. Through this approach, they developed a response classifier, which aimed to predict if a patient's untreated tumor would respond to immune checkpoint blockade or progress following treatment. This DCCN response classifier was validated in an independent cohort of 30 patients with metastatic melanoma treated at Vanderbilt-Ingram Cancer Center between 2010 and 2017.

Results: The performance of the DCCN response classifier was evaluated by calculating the area under the curve (AUC), a measure of the model's accuracy, where a score of 1 corresponds to perfect prediction. The DCCN prediction model achieved an AUC around 0.7 in both the training and validation cohorts.

To augment the prediction accuracy of the model, the researchers performed multivariable logistic regressions that combined the DCCN prediction with conventional clinical characteristics. The final model incorporated the DCCN prediction, Eastern Cooperative Oncology Group (ECOG) performance status, and treatment regimen (either anti-CTLA-4 monotherapy, anti-PD-1 monotherapy, or combination therapy). In both the training and validation cohorts, the multivariable classifier achieved an AUC around 0.8. In the validation cohort, the classifier could stratify patients into high versus low risk for disease progression, with significantly different progression-free survival outcomes between the two groups.

While the majority of patients in the training cohort received anti-CTLA-4 monotherapy (roughly 64 percent of patients), the majority of patients in the validation cohort received anti-PD-1 agents (roughly 53 percent of patients). The results suggested that some predictive patterns are not specific to the immune checkpoint target, Osman noted. Class activation mapping, which can identify regions within the digital images that the neural network uses to generate predictions, suggested that cell nuclei were important for the DCCN predictions, where larger and more numerous nuclei correlated with progression of disease. "These results suggest that ploidy might be one of the biologic determinants detected by DCCN," she added.

Author Comments: "There is potential for using computer algorithms to analyze histology images and predict treatment response, but more work needs to be done using larger training and testing datasets, along with additional validation parameters, in order to determine whether an algorithm can be developed that achieves clinical-grade performance and is broadly generalizable," Tsirigos said.

Study Limitations: Limitations of the study include the relatively small number of images used to train the computer algorithm, which included 302 images in the training cohort and 40 images in the validation cohort. "There is data to suggest that thousands of images might be needed to train models that achieve clinical-grade performance," said Tsirigos.


Funding & Disclosures: This study was sponsored by an NYU Melanoma Specialized Program of Research Excellence (SPORE) grant, the Melanoma Research Alliance, the American Cancer Society, the Laura and Isaac Perlmutter Cancer Center, and the Onassis Foundation.

Osman and Tsirigos declare no conflicts of interest.

About the American Association for Cancer Research

Founded in 1907, the American Association for Cancer Research (AACR) is the world's first and largest professional organization dedicated to advancing cancer research and its mission to prevent and cure cancer. AACR membership includes 47,000 laboratory, translational, and clinical researchers; population scientists; other health care professionals; and patient advocates residing in 127 countries. The AACR marshals the full spectrum of expertise of the cancer community to accelerate progress in the prevention, diagnosis, and treatment of cancer by annually convening more than 30 conferences and educational workshops--the largest of which is the AACR Annual Meeting, with more than 100,000 attendees for the 2020 virtual meetings and more than 22,500 attendees for past in-person meetings. In addition, the AACR publishes nine prestigious, peer-reviewed scientific journals and a magazine for cancer survivors, patients, and their caregivers. The AACR funds meritorious research directly as well as in cooperation with numerous cancer organizations. As the Scientific Partner of Stand Up To Cancer, the AACR provides expert peer review, grants administration, and scientific oversight of team science and individual investigator grants in cancer research that have the potential for near-term patient benefit. The AACR actively communicates with legislators and other policymakers about the value of cancer research and related biomedical science in saving lives from cancer. For more information about the AACR, visit

To interview Iman Osman or Aristotelis Tsirigos, please contact Richard Lobb at or 215-906-3322.

Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.