Benchmarking Computational Pathology Foundation Models For Semantic Segmentation

TL;DR

This paper benchmarks 10 foundation models for pixel-level histopathology segmentation, showing that ensemble features from multiple models improve accuracy and generalization across diverse datasets.

Contribution

It introduces a novel, model-agnostic benchmarking method using attention maps and XGBoost for efficient evaluation of foundation models in histopathology segmentation.

Findings

CONCH outperforms other models in segmentation accuracy

Ensembling features from multiple models improves performance

Models trained on different cohorts capture complementary features

Abstract

In recent years, foundation models such as CLIP, DINO,and CONCH have demonstrated remarkable domain generalization and unsupervised feature extraction capabilities across diverse imaging tasks. However, systematic and independent evaluations of these models for pixel-level semantic segmentation in histopathology remain scarce. In this study, we propose a robust benchmarking approach to asses 10 foundational models on four histopathological datasets covering both morphological tissue-region and cellular/nuclear segmentation tasks. Our method leverages attention maps of foundation models as pixel-wise features, which are then classified using a machine learning algorithm, XGBoost, enabling fast, interpretable, and model-agnostic evaluation without finetuning. We show that the vision language foundation model, CONCH performed the best across datasets when compared to vision-only foundation models, with PathDino as close second. Further analysis shows that models trained on distinct histopathology cohorts capture complementary morphological representations, and concatenating their features yields superior segmentation performance. Concatenating features from CONCH, PathDino and CellViT outperformed individual models across all the datasets by 7.95% (averaged across the datasets), suggesting that ensembles of foundation models can better generalize to diverse histopathological segmentation tasks.