Understanding Silent Failures in Medical Image Classification

TL;DR

This paper analyzes the effectiveness of confidence scoring functions in preventing silent failures in medical image classification under distribution shifts, revealing current limitations and introducing a visualization tool for deeper understanding.

Contribution

It provides the first comprehensive comparison of CSFs in medical imaging under distribution shifts and introduces SF-Visuals, an interactive tool for analyzing failures and shifts.

Findings

None of the benchmarked CSFs reliably prevent silent failures.

Distribution shifts significantly impact classifier reliability.

SF-Visuals helps visualize and understand failure causes.

Abstract

To ensure the reliable use of classification systems in medical applications, it is crucial to prevent silent failures. This can be achieved by either designing classifiers that are robust enough to avoid failures in the first place, or by detecting remaining failures using confidence scoring functions (CSFs). A predominant source of failures in image classification is distribution shifts between training data and deployment data. To understand the current state of silent failure prevention in medical imaging, we conduct the first comprehensive analysis comparing various CSFs in four biomedical tasks and a diverse range of distribution shifts. Based on the result that none of the benchmarked CSFs can reliably prevent silent failures, we conclude that a deeper understanding of the root causes of failures in the data is required. To facilitate this, we introduce SF-Visuals, an interactive analysis tool that uses latent space clustering to visualize shifts and failures. On the basis of various examples, we demonstrate how this tool can help researchers gain insight into the requirements for safe application of classification systems in the medical domain. The open-source benchmark and tool are at: https://github.com/IML-DKFZ/sf-visuals.