Deep Learning-based Prediction of Breast Cancer Tumor and Immune Phenotypes from Histopathology

TL;DR

This study develops deep learning models using histopathology images to predict tumor and immune phenotypes in breast cancer, aiming to improve personalized treatment strategies.

Contribution

It introduces multiple instance learning algorithms with advanced feature extraction to accurately assess tumor microenvironment phenotypes from H&E slides.

Findings

Models achieved AUROC scores above 0.70 for most pathways.

Attention maps identified biologically relevant spatial patterns.

First application of deep learning for TME phenotype prediction from histopathology.

Abstract

The interactions between tumor cells and the tumor microenvironment (TME) dictate therapeutic efficacy of radiation and many systemic therapies in breast cancer. However, to date, there is not a widely available method to reproducibly measure tumor and immune phenotypes for each patient's tumor. Given this unmet clinical need, we applied multiple instance learning (MIL) algorithms to assess activity of ten biologically relevant pathways from the hematoxylin and eosin (H&E) slide of primary breast tumors. We employed different feature extraction approaches and state-of-the-art model architectures. Using binary classification, our models attained area under the receiver operating characteristic (AUROC) scores above 0.70 for nearly all gene expression pathways and on some cases, exceeded 0.80. Attention maps suggest that our trained models recognize biologically relevant spatial patterns of cell sub-populations from H&E. These efforts represent a first step towards developing computational H&E biomarkers that reflect facets of the TME and hold promise for augmenting precision oncology.