Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning

TL;DR

This paper introduces a novel dual-stream MIL network for whole slide image classification that leverages self-supervised contrastive learning and multiscale fusion, achieving high accuracy without localized annotations.

Contribution

The paper presents a new MIL aggregator with trainable relation modeling, integrates self-supervised contrastive learning for large unbalanced bags, and employs pyramidal multiscale fusion, advancing WSI classification methods.

Findings

Achieves less than 2% accuracy gap compared to fully-supervised methods.

Outperforms previous MIL-based approaches on WSI datasets.

Demonstrates superior generalization on standard MIL benchmarks.

Abstract

We address the challenging problem of whole slide image (WSI) classification. WSIs have very high resolutions and usually lack localized annotations. WSI classification can be cast as a multiple instance learning (MIL) problem when only slide-level labels are available. We propose a MIL-based method for WSI classification and tumor detection that does not require localized annotations. Our method has three major components. First, we introduce a novel MIL aggregator that models the relations of the instances in a dual-stream architecture with trainable distance measurement. Second, since WSIs can produce large or unbalanced bags that hinder the training of MIL models, we propose to use self-supervised contrastive learning to extract good representations for MIL and alleviate the issue of prohibitive memory cost for large bags. Third, we adopt a pyramidal fusion mechanism for multiscale WSI features, and further improve the accuracy of classification and localization. Our model is evaluated on two representative WSI datasets. The classification accuracy of our model compares favorably to fully-supervised methods, with less than 2% accuracy gap across datasets. Our results also outperform all previous MIL-based methods. Additional benchmark results on standard MIL datasets further demonstrate the superior performance of our MIL aggregator on general MIL problems. GitHub repository: https://github.com/binli123/dsmil-wsi