AdaFusion: Prompt-Guided Inference with Adaptive Fusion of Pathology Foundation Models

TL;DR

AdaFusion introduces a prompt-guided inference framework that adaptively fuses multiple pathology foundation models, improving performance and interpretability in histopathology tasks by leveraging complementary knowledge.

Contribution

It is among the first to dynamically integrate multiple PFMs using prompt-guided adaptive fusion, enhancing generalisability and interpretability in pathology applications.

Findings

Consistently outperforms individual PFMs on various benchmarks.

Provides interpretable insights into model-specific biosemantic specialization.

Enhances performance in classification and regression tasks.

Abstract

Pathology foundation models (PFMs) have demonstrated strong representational capabilities through self-supervised pre-training on large-scale, unannotated histopathology image datasets. However, their diverse yet opaque pretraining contexts, shaped by both data-related and structural/training factors, introduce latent biases that hinder generalisability and transparency in downstream applications. In this paper, we propose AdaFusion, a novel prompt-guided inference framework that, to our knowledge, is among the very first to dynamically integrate complementary knowledge from multiple PFMs. Our method compresses and aligns tile-level features from diverse models and employs a lightweight attention mechanism to adaptively fuse them based on tissue phenotype context. We evaluate AdaFusion on three real-world benchmarks spanning treatment response prediction, tumour grading, and spatial gene expression inference. Our approach consistently surpasses individual PFMs across both classification and regression tasks, while offering interpretable insights into each model's biosemantic specialisation. These results highlight AdaFusion's ability to bridge heterogeneous PFMs, achieving both enhanced performance and interpretability of model-specific inductive biases.