Large language models enable multi-modality integration for brain tumor diagnosis and prognosis

Brain tumors, particularly gliomas, remain challenging to diagnose and treat. While radiology reports can offer non-invasive insights into tumor size, shape, and progression, with pathology reports providing definitive evidence of cellular and molecular features, the lack of integration between these two information sources often complicates clinical decision-making.

To overcome this, a research team led by Dr. Zhuoqi Ma from the Department of Radiology at Brown University and Brown University Health created a pipeline that employs a pre-trained large language model to unify radiology and pathology reports.

In a sample of 426 patients, the system achieved a micro F1-score of 0.849 for tumor presence and 0.929 for tumor stability, surpassing single-source methods by more than 10%.

“Large language models  (LLM) can synthesize information from multiple domains and deliver a more complete picture of tumor status,” shares Ma. “This not only improves diagnostic accuracy but also allows the model to predict survival outcomes without additional training.”

The team further validated their approach in an independent cohort of glioblastoma patients. The model’s predictions, particularly those related to tumor stability, significantly distinguished high-risk from low-risk groups (p=0.017), demonstrating prognostic value comparable to well-established biomarkers such as MGMT methylation status. These findings highlight the potential of LLM-based integration to reduce diagnostic uncertainty and support more personalized treatment strategies in neuro-oncology.

The study, published in the KeAi journal Meta-Radiology, establishes a foundation for future research on multi-source integration in clinical oncology.

“Expanding the approach to include modalities such as MRI imaging and genomic profiles could further enhance predictive power and accelerate progress toward precision cancer care,” adds Ma. 

###

Contact the author: Zhicheng Jiao, PhD, Brown University Health, zhicheng_jiao@brown.edu

The publisher KeAi was established by Elsevier and China Science Publishing & Media Ltd to unfold quality research globally. In 2013, our focus shifted to open access publishing. We now proudly publish more than 200 world-class, open access, English language journals, spanning all scientific disciplines. Many of these are titles we publish in partnership with prestigious societies and academic institutions, such as the National Natural Science Foundation of China (NSFC).