EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning

TL;DR

EviCare is a novel framework that enhances diagnosis prediction from electronic health records by integrating deep model guidance with large language models for better accuracy and interpretability, especially for novel conditions.

Contribution

EviCare introduces an in-context reasoning approach combining deep model inference, evidential prioritization, and relational evidence construction to improve diagnosis prediction with LLMs.

Findings

EviCare outperforms LLM-only and deep model-only baselines by 20.65% on average.

Significant improvements in predicting novel diagnoses, averaging 30.97%.

Demonstrates effectiveness on MIMIC-III and MIMIC-IV benchmarks.

Abstract

Recent advances in large language models (LLMs) have enabled promising progress in diagnosis prediction from electronic health records (EHRs). However, existing LLM-based approaches tend to overfit to historically observed diagnoses, often overlooking novel yet clinically important conditions that are critical for early intervention. To address this, we propose EviCare, an in-context reasoning framework that integrates deep model guidance into LLM-based diagnosis prediction. Rather than prompting LLMs directly with raw EHR inputs, EviCare performs (1) deep model inference for candidate selection, (2) evidential prioritization for set-based EHRs, and (3) relational evidence construction for novel diagnosis prediction. These signals are then composed into an adaptive in-context prompt to guide LLM reasoning in an accurate and interpretable manner. Extensive experiments on two real-world EHR benchmarks (MIMIC-III and MIMIC-IV) demonstrate that EviCare achieves significant performance gains, which consistently outperforms both LLM-only and deep model-only baselines by an average of 20.65\% across precision and accuracy metrics. The improvements are particularly notable in challenging novel diagnosis prediction, yielding average improvements of 30.97\%.