AI that reads MRI, pathology slides, and clinical data together predicts breast cancer chemo response before treatment starts

For many breast cancer patients, the most consequential question of neoadjuvant chemotherapy — will this tumor melt away completely? — can be answered with certainty only after surgery, when pathologists examine what remains. A study published in The Innovation Oncology reports an artificial intelligence framework, BC-SMART, that brings that answer forward to before the first infusion, by combining three kinds of data doctors already collect. Pathological complete response, or pCR — the disappearance of invasive cancer from breast and lymph nodes after chemotherapy — is one of the strongest predictors of long-term survival in patients who receive neoadjuvant chemotherapy. But responses vary widely between patients, and the information available beforehand is scattered: radiologists read MRI scans, pathologists examine biopsy slides, and clinicians weigh tumor size, grade, and receptor status, usually in separate steps. A team led by researchers at Guangzhou First People's Hospital compiled a retrospective cohort of 705 women with breast cancer from four Chinese institutions, divided into training, internal test, and external test sets. BC-SMART processes each data stream with a model suited to its format — a slice-wise attention network for dynamic contrast-enhanced MRI and a weakly supervised multiple-instance learning model for hematoxylin-eosin-stained whole-slide biopsy images — then combines modality scores with clinicopathologic variables through a transparent logistic-regression fusion layer. In testing, the framework achieved areas under the receiver operating characteristic curve of 0.903 in the internal cohort and 0.901 in the external cohort, with accuracies of 0.872 and 0.829, respectively. It outperformed models built on clinical variables, MRI, or pathology alone. Attention heatmaps and Shapley value analyses highlighted tumor cores, peritumoral tissue, and tumor-stroma interfaces, and quantified each modality's contribution — an audit trail the authors argue is essential for clinical trust. To test usefulness in practice, the team built a prototype web interface and ran a multidisciplinary reader study. With BC-SMART's support, both junior and senior teams improved their predictive performance and reported greater confidence, positioning the tool as a "second reader" rather than a replacement for clinical judgment. The authors are careful about limits. The study is retrospective; the model predicts only pretreatment response and does not ingest on-treatment imaging; the pathology branch learned from core-needle biopsies, which sample only part of a tumor; and no molecular profiling validated the learned features. Prospective validation in real clinical workflows, extension to targeted and immunotherapy regimens, and strategies for missing modalities are next. If those steps succeed, the payoff is timing: identifying likely non-responders before chemotherapy starts could spare patients months of toxic treatment and open a window to switch strategies — or optimize surgery — earlier.