Deep Learning-Based Prediction of Molecular Tumor Biomarkers from H&E: A Practical Review

TL;DR

This review discusses how deep learning models applied to H&E histology images can predict molecular tumor biomarkers, offering a cost-effective alternative to traditional molecular testing and enabling new insights into tumor heterogeneity and patient prognosis.

Contribution

It provides a comprehensive overview of recent deep learning methods for predicting molecular biomarkers from H&E images, highlighting trends, challenges, and potential clinical applications.

Findings

Deep learning models can predict various molecular biomarkers from H&E images.

Weakly supervised approaches are prevalent and effective in this domain.

Challenges include limited training data, validation rigor, and model interpretability.

Abstract

Molecular and genomic properties are critical in selecting cancer treatments to target individual tumors, particularly for immunotherapy. However, the methods to assess such properties are expensive, time-consuming, and often not routinely performed. Applying machine learning to H&E images can provide a more cost-effective screening method. Dozens of studies over the last few years have demonstrated that a variety of molecular biomarkers can be predicted from H&E alone using the advancements of deep learning: molecular alterations, genomic subtypes, protein biomarkers, and even the presence of viruses. This article reviews the diverse applications across cancer types and the methodology to train and validate these models on whole slide images. From bottom-up to pathologist-driven to hybrid approaches, the leading trends include a variety of weakly supervised deep learning-based approaches, as well as mechanisms for training strongly supervised models in select situations. While results of these algorithms look promising, some challenges still persist, including small training sets, rigorous validation, and model explainability. Biomarker prediction models may yield a screening method to determine when to run molecular tests or an alternative when molecular tests are not possible. They also create new opportunities in quantifying intratumoral heterogeneity and predicting patient outcomes.