An Empirical Analysis of Calibration and Selective Prediction in Multimodal Clinical Condition Classification

TL;DR

This study empirically examines the reliability of uncertainty-based selective prediction in multimodal clinical condition classification, revealing calibration issues that impair safety guarantees.

Contribution

It identifies a task-specific failure mode of selective prediction due to class-dependent miscalibration in multimodal clinical AI models.

Findings

Selective prediction can degrade performance despite strong metrics.

Models exhibit severe class-dependent miscalibration, especially for underrepresented conditions.

Standard aggregate metrics may hide these calibration issues.

Abstract

As artificial intelligence systems move toward clinical deployment, ensuring reliable prediction behavior is fundamental for safety-critical decision-making tasks. One proposed safeguard is selective prediction, where models can defer uncertain predictions to human experts for review. In this work, we empirically evaluate the reliability of uncertainty-based selective prediction in multilabel clinical condition classification using multimodal ICU data. Across a range of state-of-the-art unimodal and multimodal models, we find that selective prediction can substantially degrade performance despite strong standard evaluation metrics. This failure is driven by severe class-dependent miscalibration, whereby models assign high uncertainty to correct predictions and low uncertainty to incorrect ones, particularly for underrepresented clinical conditions. Our results show that commonly used aggregate metrics can obscure these effects, limiting their ability to assess selective prediction behavior in this setting. Taken together, our findings characterize a task-specific failure mode of selective prediction in multimodal clinical condition classification and highlight the need for calibration-aware evaluation to provide strong guarantees of safety and robustness in clinical AI.