Beyond accuracy: quantifying the reliability of Multiple Instance Learning for Whole Slide Image classification

TL;DR

This paper introduces three metrics to evaluate the reliability of Multiple Instance Learning models for Whole Slide Image classification, highlighting the importance of trustworthiness in clinical applications.

Contribution

It proposes quantitative reliability metrics and evaluates several MIL architectures, identifying MEAN-POOL-INS as a notably reliable and efficient model.

Findings

MEAN-POOL-INS outperforms other MIL models in reliability.

Reliability metrics reveal differences not captured by accuracy alone.

Reliability assessment is crucial for clinical deployment of MIL models.

Abstract

Machine learning models have become integral to many fields, but their reliability, defined as producing dependable, trustworthy, and domain-consistent predictions, remains a critical concern. Multiple Instance Learning (MIL) models designed for Whole Slide Image (WSI) classification in computational pathology are rarely evaluated in terms of reliability, leaving a key gap in understanding their suitability for high-stakes applications like clinical decision-making. In this paper, we address this gap by introducing three quantitative metrics for reliability assessment and applying them to several widely used MIL architectures across three region-wise annotated pathology datasets. Our findings indicate that the mean pooling instance (MEAN-POOL-INS)model demonstrates superior reliability compared to other networks, despite its simple architectural design and computational efficiency. These findings underscore the need of reliability evaluation alongside predictive performance in MIL models and establish MEAN-POOL-INS as a strong, trustworthy baseline for future research.