Vim4Path: Self-Supervised Vision Mamba for Histopathology Images

TL;DR

This paper introduces Vim, a vision architecture inspired by state space models, for self-supervised learning on histopathology images, demonstrating superior performance over ViT especially at smaller scales, and aligning better with pathologist workflows.

Contribution

The paper proposes leveraging the Vision Mamba architecture within the DINO framework for improved self-supervised representation learning in computational pathology, outperforming Vision Transformers.

Findings

Vim achieves an 8.21 higher ROC AUC than ViT at similar model sizes.

Vim performs better at smaller scales compared to ViT.

Explainability analysis shows Vim mimics pathologist workflows.

Abstract

Representation learning from Gigapixel Whole Slide Images (WSI) poses a significant challenge in computational pathology due to the complicated nature of tissue structures and the scarcity of labeled data. Multi-instance learning methods have addressed this challenge, leveraging image patches to classify slides utilizing pretrained models using Self-Supervised Learning (SSL) approaches. The performance of both SSL and MIL methods relies on the architecture of the feature encoder. This paper proposes leveraging the Vision Mamba (Vim) architecture, inspired by state space models, within the DINO framework for representation learning in computational pathology. We evaluate the performance of Vim against Vision Transformers (ViT) on the Camelyon16 dataset for both patch-level and slide-level classification. Our findings highlight Vim's enhanced performance compared to ViT, particularly at smaller scales, where Vim achieves an 8.21 increase in ROC AUC for models of similar size. An explainability analysis further highlights Vim's capabilities, which reveals that Vim uniquely emulates the pathologist workflow-unlike ViT. This alignment with human expert analysis highlights Vim's potential in practical diagnostic settings and contributes significantly to developing effective representation-learning algorithms in computational pathology. We release the codes and pretrained weights at \url{https://github.com/AtlasAnalyticsLab/Vim4Path}.