Accounting for Dependencies in Deep Learning Based Multiple Instance Learning for Whole Slide Imaging

TL;DR

This paper introduces a deep learning-based multiple instance learning method for whole slide imaging that explicitly models dependencies between image patches using self-attention transformers, achieving state-of-the-art results on a large dataset.

Contribution

The paper's main novelty is embedding self-attention transformers to capture dependencies between instances in MIL for WSIs, along with an instance-wise loss function based on pseudo-labels.

Findings

Achieved state-of-the-art performance on the PANDA dataset.

Effectively modeled dependencies between image patches.

Demonstrated improved classification accuracy over baseline methods.

Abstract

Multiple instance learning (MIL) is a key algorithm for classification of whole slide images (WSI). Histology WSIs can have billions of pixels, which create enormous computational and annotation challenges. Typically, such images are divided into a set of patches (a bag of instances), where only bag-level class labels are provided. Deep learning based MIL methods calculate instance features using convolutional neural network (CNN). Our proposed approach is also deep learning based, with the following two contributions: Firstly, we propose to explicitly account for dependencies between instances during training by embedding self-attention Transformer blocks to capture dependencies between instances. For example, a tumor grade may depend on the presence of several particular patterns at different locations in WSI, which requires to account for dependencies between patches. Secondly, we propose an instance-wise loss function based on instance pseudo-labels. We compare the proposed algorithm to multiple baseline methods, evaluate it on the PANDA challenge dataset, the largest publicly available WSI dataset with over 11K images, and demonstrate state-of-the-art results.