Fusion of Multi-scale Heterogeneous Pathology Foundation Models for Whole Slide Image Analysis

TL;DR

This paper introduces FuseCPath, a novel framework that effectively fuses multi-scale heterogeneous pathology foundation models for whole slide image analysis, significantly improving performance in computational pathology tasks.

Contribution

The paper presents a multi-view clustering, cluster-level re-embedding, and collaborative distillation approach to fuse diverse pathology FMs, addressing heterogeneity and enhancing ensemble performance.

Findings

FuseCPath achieves state-of-the-art results across multiple datasets.

The framework effectively filters discriminative patches using multi-view clustering.

It improves the integration of slide-level features through collaborative distillation.

Abstract

Whole slide image (WSI) analysis has emerged as an increasingly essential technique in computational pathology. Recent advances in the pathology foundation models (FMs) have demonstrated significant advantages in deriving meaningful patch-level or slide-level multi-scale features from WSIs. However, current pathology FMs have exhibited substantial heterogeneity caused by diverse private training datasets and different network architectures. This heterogeneity introduces performance variability when we utilize the features from different FMs in the downstream tasks. To fully explore the advantages of multiple FMs effectively, in this work, we propose a novel framework for the fusion of multi-scale heterogeneous pathology FMs, called FuseCPath, yielding a model with a superior ensemble performance. The main contributions of our framework can be summarized as follows: (i) To guarantee the representativeness of the training patches, we propose a multi-view clustering-based method to filter out the discriminative patches via multiple FMs' embeddings. (ii) To effectively fuse the patch-level FMs, we devise a cluster-level re-embedding strategy to online capture patch-level local features. (iii) To effectively fuse the slide-level FMs, we devise a collaborative distillation strategy to explore the connections between slide-level FMs. Extensive experiments demonstrate that the proposed FuseCPath achieves state-of-the-art performance across multiple tasks on diverse datasets.