From Pixels to Gigapixels: Bridging Local Inductive Bias and Long-Range Dependencies with Pixel-Mamba

TL;DR

Pixel-Mamba is a novel deep learning architecture that efficiently processes gigapixel whole slide images by combining local inductive biases with long-range dependencies, achieving state-of-the-art results without extensive pretraining.

Contribution

Introduces Pixel-Mamba, a new model that hierarchically integrates local and global information in gigapixel WSIs using a state-space model with linear memory complexity.

Findings

Achieves or surpasses SOTA performance on tumor staging and survival analysis.

Does not require pathology-specific pretraining.

Demonstrates high efficiency and effectiveness in end-to-end WSI analysis.

Abstract

Histopathology plays a critical role in medical diagnostics, with whole slide images (WSIs) offering valuable insights that directly influence clinical decision-making. However, the large size and complexity of WSIs may pose significant challenges for deep learning models, in both computational efficiency and effective representation learning. In this work, we introduce Pixel-Mamba, a novel deep learning architecture designed to efficiently handle gigapixel WSIs. Pixel-Mamba leverages the Mamba module, a state-space model (SSM) with linear memory complexity, and incorporates local inductive biases through progressively expanding tokens, akin to convolutional neural networks. This enables Pixel-Mamba to hierarchically combine both local and global information while efficiently addressing computational challenges. Remarkably, Pixel-Mamba achieves or even surpasses the quantitative performance of state-of-the-art (SOTA) foundation models that were pretrained on millions of WSIs or WSI-text pairs, in a range of tumor staging and survival analysis tasks, {\bf even without requiring any pathology-specific pretraining}. Extensive experiments demonstrate the efficacy of Pixel-Mamba as a powerful and efficient framework for end-to-end WSI analysis.