Context-dependent Explainability and Contestability for Trustworthy Medical Artificial Intelligence: Misclassification Identification of Morbidity Recognition Models in Preterm Infants

TL;DR

This paper introduces a novel explainability methodology for tabular data in medical AI, enabling clinicians to identify model errors in preterm infant morbidity recognition, thereby enhancing trust and decision-making.

Contribution

It proposes a new approach combining clinical context, global explanations, and latent space similarity to improve model failure detection in medical tabular data.

Findings

Successfully identified misclassification cases in morbidity recognition models.

Provided actionable insights to clinicians for informed decision-making.

Enhanced trustworthiness of AI models in clinical settings.

Abstract

Although machine learning (ML) models of AI achieve high performances in medicine, they are not free of errors. Empowering clinicians to identify incorrect model recommendations is crucial for engendering trust in medical AI. Explainable AI (XAI) aims to address this requirement by clarifying AI reasoning to support the end users. Several studies on biomedical imaging achieved promising results recently. Nevertheless, solutions for models using tabular data are not sufficient to meet the requirements of clinicians yet. This paper proposes a methodology to support clinicians in identifying failures of ML models trained with tabular data. We built our methodology on three main pillars: decomposing the feature set by leveraging clinical context latent space, assessing the clinical association of global explanations, and Latent Space Similarity (LSS) based local explanations. We demonstrated our methodology on ML-based recognition of preterm infant morbidities caused by infection. The risk of mortality, lifelong disability, and antibiotic resistance due to model failures was an open research question in this domain. We achieved to identify misclassification cases of two models with our approach. By contextualizing local explanations, our solution provides clinicians with actionable insights to support their autonomy for informed final decisions.