Multi-modal Knowledge Decomposition based Online Distillation for Biomarker Prediction in Breast Cancer Histopathology

TL;DR

This paper introduces a multi-modal knowledge decomposition method for online distillation that improves biomarker prediction in breast cancer histopathology, enabling effective use of single-modality data during inference.

Contribution

It proposes a novel online distillation framework with multi-modal knowledge decomposition, similarity-preserving knowledge distillation, and collaborative learning for improved biomarker prediction.

Findings

Achieves superior biomarker prediction performance with single-modality data.

Effectively leverages paired genomic-pathology data during training.

Demonstrates robustness across multiple datasets.

Abstract

Immunohistochemical (IHC) biomarker prediction benefits from multi-modal data fusion analysis. However, the simultaneous acquisition of multi-modal data, such as genomic and pathological information, is often challenging due to cost or technical limitations. To address this challenge, we propose an online distillation approach based on Multi-modal Knowledge Decomposition (MKD) to enhance IHC biomarker prediction in haematoxylin and eosin (H\&E) stained histopathology images. This method leverages paired genomic-pathology data during training while enabling inference using either pathology slides alone or both modalities. Two teacher and one student models are developed to extract modality-specific and modality-general features by minimizing the MKD loss. To maintain the internal structural relationships between samples, Similarity-preserving Knowledge Distillation (SKD) is applied. Additionally, Collaborative Learning for Online Distillation (CLOD) facilitates mutual learning between teacher and student models, encouraging diverse and complementary learning dynamics. Experiments on the TCGA-BRCA and in-house QHSU datasets demonstrate that our approach achieves superior performance in IHC biomarker prediction using uni-modal data. Our code is available at https://github.com/qiyuanzz/MICCAI2025_MKD.