Ordinal Multiple-instance Learning for Ulcerative Colitis Severity Estimation with Selective Aggregated Transformer

TL;DR

This paper introduces a novel transformer-based multiple-instance learning approach for patient-level ulcerative colitis severity estimation, effectively utilizing multiple images per patient to improve accuracy over existing image-level methods.

Contribution

It proposes a selective aggregator transformer for ordinal multiple-instance learning, enabling better aggregation of severe features from multiple images in clinical settings.

Findings

Outperforms state-of-the-art MIL methods on two datasets.

Demonstrates superior accuracy in real clinical settings.

Effectively distinguishes between adjacent severity classes.

Abstract

Patient-level diagnosis of severity in ulcerative colitis (UC) is common in real clinical settings, where the most severe score in a patient is recorded. However, previous UC classification methods (i.e., image-level estimation) mainly assumed the input was a single image. Thus, these methods can not utilize severity labels recorded in real clinical settings. In this paper, we propose a patient-level severity estimation method by a transformer with selective aggregator tokens, where a severity label is estimated from multiple images taken from a patient, similar to a clinical setting. Our method can effectively aggregate features of severe parts from a set of images captured in each patient, and it facilitates improving the discriminative ability between adjacent severity classes. Experiments demonstrate the effectiveness of the proposed method on two datasets compared with the state-of-the-art MIL methods. Moreover, we evaluated our method in real clinical settings and confirmed that our method outperformed the previous image-level methods. The code is publicly available at https://github.com/Shiku-Kaito/Ordinal-Multiple-instance-Learning-for-Ulcerative-Colitis-Severity-Estimation.