RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification

TL;DR

This paper introduces a RoFormer-based module for position-aware multiple instance learning, enabling efficient spatial modeling of tissue patches in whole slide images, and outperforms existing methods on key datasets.

Contribution

The paper presents a novel RoFormer layer with relative positional encoding for MIL in WSIs, addressing computational challenges and spatial correlation modeling.

Findings

Outperforms state-of-the-art MIL models on three datasets

Enables full self-attention with relative position encoding on large WSIs

Efficiently models tissue structure and patch correlation

Abstract

Whole slide image (WSI) classification is a critical task in computational pathology. However, the gigapixel-size of such images remains a major challenge for the current state of deep-learning. Current methods rely on multiple-instance learning (MIL) models with frozen feature extractors. Given the the high number of instances in each image, MIL methods have long assumed independence and permutation-invariance of patches, disregarding the tissue structure and correlation between patches. Recent works started studying this correlation between instances but the computational workload of such a high number of tokens remained a limiting factor. In particular, relative position of patches remains unaddressed. We propose to apply a straightforward encoding module, namely a RoFormer layer , relying on memory-efficient exact self-attention and relative positional encoding. This module can perform full self-attention with relative position encoding on patches of large and arbitrary shaped WSIs, solving the need for correlation between instances and spatial modeling of tissues. We demonstrate that our method outperforms state-of-the-art MIL models on three commonly used public datasets (TCGA-NSCLC, BRACS and Camelyon16)) on weakly supervised classification tasks. Code is available at https://github.com/Sanofi-Public/DDS-RoFormerMIL