Jigsaw Regularization in Whole-Slide Image Classification

TL;DR

This paper introduces a novel graph neural network approach with jigsaw regularization and foundation-model embeddings to improve whole-slide image classification in computational pathology, leveraging spatial structure more effectively.

Contribution

It proposes a new method combining foundation-model embeddings and graph neural networks with jigsaw regularization for spatially aware MIL in pathology images.

Findings

Significant accuracy improvement over state-of-the-art methods.

Effective incorporation of local and across-patch spatial information.

Validated on multiple cancer datasets.

Abstract

Computational pathology involves the digitization of stained tissues into whole-slide images (WSIs) that contain billions of pixels arranged as contiguous patches. Statistical analysis of WSIs largely focuses on classification via multiple instance learning (MIL), in which slide-level labels are inferred from unlabeled patches. Most MIL methods treat patches as exchangeable, overlooking the rich spatial and topological structure that underlies tissue images. This work builds on recent graph-based methods that aim to incorporate spatial awareness into MIL. Our approach is new in two regards: (1) we deploy vision \emph{foundation-model embeddings} to incorporate local spatial structure within each patch, and (2) achieve across-patch spatial awareness using graph neural networks together with a novel {\em jigsaw regularization}. We find that a combination of these two features markedly improves classification over state-of-the-art attention-based MIL approaches on benchmark datasets in breast, head-and-neck, and colon cancer.