Enabling Progressive Whole-slide Image Analysis with Multi-scale Pyramidal Network

TL;DR

This paper introduces MSPN, a plug-and-play multi-scale pyramidal network that enhances attention-based MIL for whole-slide image analysis by providing progressive multi-scale context, improving performance across multiple clinical tasks.

Contribution

The paper presents MSPN, a novel multi-scale pyramidal network that integrates with attention-based MIL to enable progressive analysis of whole-slide images, addressing limitations of previous multi-scale methods.

Findings

MSPN improves MIL performance across various tasks.

MSPN is lightweight and easy to integrate.

MSPN enhances feature learning across scales.

Abstract

Multiple-instance Learning (MIL) is commonly used to undertake computational pathology (CPath) tasks, and the use of multi-scale patches allows diverse features across scales to be learned. Previous studies using multi-scale features in clinical applications rely on multiple inputs across magnifications with late feature fusion, which does not retain the link between features across scales while the inputs are dependent on arbitrary, manufacturer-defined magnifications, being inflexible and computationally expensive. In this paper, we propose the Multi-scale Pyramidal Network (MSPN), which is plug-and-play over attention-based MIL that introduces progressive multi-scale analysis on WSI. Our MSPN consists of (1) grid-based remapping that uses high magnification features to derive coarse features and (2) the coarse guidance network (CGN) that learns coarse contexts. We benchmark MSPN as an add-on module to 4 attention-based frameworks using 4 clinically relevant tasks across 3 types of foundation model, as well as the pre-trained MIL framework. We show that MSPN consistently improves MIL across the compared configurations and tasks, while being lightweight and easy-to-use.