Holistic Artificial Intelligence in Medicine; improved performance and explainability

TL;DR

This paper introduces xHAIM, an advanced framework that enhances AI performance and explainability in medicine by integrating generative AI to produce patient summaries and clinical explanations, thereby improving predictive accuracy and clinical utility.

Contribution

The paper presents xHAIM, a novel framework that combines generative AI with multimodal data to improve prediction accuracy and provide explainability in medical AI applications.

Findings

xHAIM improves average AUC from 79.9% to 90.3%.

Provides interactive, patient-specific clinical explanations.

Transforms AI into an explainable decision support system.

Abstract

With the increasing interest in deploying Artificial Intelligence in medicine, we previously introduced HAIM (Holistic AI in Medicine), a framework that fuses multimodal data to solve downstream clinical tasks. However, HAIM uses data in a task-agnostic manner and lacks explainability. To address these limitations, we introduce xHAIM (Explainable HAIM), a novel framework leveraging Generative AI to enhance both prediction and explainability through four structured steps: (1) automatically identifying task-relevant patient data across modalities, (2) generating comprehensive patient summaries, (3) using these summaries for improved predictive modeling, and (4) providing clinical explanations by linking predictions to patient-specific medical knowledge. Evaluated on the HAIM-MIMIC-MM dataset, xHAIM improves average AUC from 79.9% to 90.3% across chest pathology and operative tasks. Importantly, xHAIM transforms AI from a black-box predictor into an explainable decision support system, enabling clinicians to interactively trace predictions back to relevant patient data, bridging AI advancements with clinical utility.