Unsupervised Foundation Model-Agnostic Slide-Level Representation Learning

TL;DR

This paper introduces COBRA, a contrastive self-supervised learning method that integrates multiple foundation models to generate effective slide-level representations from WSIs, outperforming existing methods on multiple cohorts.

Contribution

The paper presents a novel SSL approach, COBRA, that combines multiple FMs in feature space for slide representation learning, improving performance and compatibility.

Findings

COBRA exceeds state-of-the-art slide encoders by at least +4.4% AUC on CPTAC cohorts.

Pretrained on only 3048 WSIs from TCGA, COBRA generalizes well to unseen feature extractors.

Code for COBRA is publicly available at GitHub.

Abstract

Representation learning of pathology whole-slide images (WSIs) has primarily relied on weak supervision with Multiple Instance Learning (MIL). This approach leads to slide representations highly tailored to a specific clinical task. Self-supervised learning (SSL) has been successfully applied to train histopathology foundation models (FMs) for patch embedding generation. However, generating patient or slide level embeddings remains challenging. Existing approaches for slide representation learning extend the principles of SSL from patch level learning to entire slides by aligning different augmentations of the slide or by utilizing multimodal data. By integrating tile embeddings from multiple FMs, we propose a new single modality SSL method in feature space that generates useful slide representations. Our contrastive pretraining strategy, called COBRA, employs multiple FMs and an architecture based on Mamba-2. COBRA exceeds performance of state-of-the-art slide encoders on four different public Clinical Protemic Tumor Analysis Consortium (CPTAC) cohorts on average by at least +4.4% AUC, despite only being pretrained on 3048 WSIs from The Cancer Genome Atlas (TCGA). Additionally, COBRA is readily compatible at inference time with previously unseen feature extractors. Code available at https://github.com/KatherLab/COBRA.