Cracking Instance Jigsaw Puzzles: An Alternative to Multiple Instance Learning for Whole Slide Image Analysis

TL;DR

This paper introduces a novel method for whole slide image analysis that restores the order of shuffled instances to better capture spatial correlations, outperforming traditional multiple instance learning approaches.

Contribution

It proposes a new approach using instance jigsaw puzzles and Siamese networks, offering a non-permutation-invariant alternative to MIL for improved WSI analysis.

Findings

Outperforms recent MIL competitors in classification tasks

Effective in survival prediction tasks

Validated on multiple WSI datasets

Abstract

While multiple instance learning (MIL) has shown to be a promising approach for histopathological whole slide image (WSI) analysis, its reliance on permutation invariance significantly limits its capacity to effectively uncover semantic correlations between instances within WSIs. Based on our empirical and theoretical investigations, we argue that approaches that are not permutation-invariant but better capture spatial correlations between instances can offer more effective solutions. In light of these findings, we propose a novel alternative to existing MIL for WSI analysis by learning to restore the order of instances from their randomly shuffled arrangement. We term this task as cracking an instance jigsaw puzzle problem, where semantic correlations between instances are uncovered. To tackle the instance jigsaw puzzles, we propose a novel Siamese network solution, which is theoretically justified by optimal transport theory. We validate the proposed method on WSI classification and survival prediction tasks, where the proposed method outperforms the recent state-of-the-art MIL competitors. The code is available at https://github.com/xiwenc1/MIL-JigsawPuzzles.