EHR-R1: A Reasoning-Enhanced Foundational Language Model for Electronic Health Record Analysis

TL;DR

This paper introduces EHR-R1, a reasoning-enhanced language model tailored for electronic health record analysis, supported by a large reasoning dataset and a new benchmark, significantly improving clinical decision-making capabilities.

Contribution

The paper presents EHR-Ins, a large EHR reasoning dataset, and EHR-R1, a series of reasoning-enhanced LLMs, along with EHR-Bench for comprehensive evaluation, advancing EHR analysis technology.

Findings

EHR-R1 outperforms GPT-4o by over 30 points on MIMIC-Bench.

EHR-R1 achieves 10% higher zero-shot AUROC on EHRSHOT.

The framework enables scalable high-quality reasoning data generation.

Abstract

Electronic Health Records (EHRs) contain rich yet complex information, and their automated analysis is critical for clinical decision-making. Despite recent advances of large language models (LLMs) in clinical workflows, their ability to analyze EHRs remains limited due to narrow task coverage and lack of EHR-oriented reasoning capabilities. This paper aims to bridge the gap, specifically, we present EHR-Ins, a large-scale, comprehensive EHR reasoning instruction dataset, comprising 300k high-quality reasoning cases and 4M non-reasoning cases across 42 distinct EHR tasks. Its core innovation is a thinking-graph-driven framework that enables to generate high-quality reasoning data at scale. Based on it, we develop EHR-R1, a series of reasoning-enhanced LLMs with up to 72B parameters tailored for EHR analysis. Through a multi-stage training paradigm, including domain adaptation, reasoning enhancement, and reinforcement learning, EHR-R1 systematically acquires domain knowledge and diverse reasoning capabilities, enabling accurate and robust EHR analysis. Lastly, we introduce EHR-Bench, a new benchmark curated from MIMIC-IV, spanning 42 tasks, to comprehensively assess reasoning and prediction across EHR scenarios. In experiments, we show that the resulting EHR-R1 consistently outperforms state-of-the-art commercial and open-source LLMs (including DeepSeek-V3 and GPT-4o), surpassing GPT-4o by over 30 points on MIMIC-Bench and achieving a 10\% higher zero-shot AUROC on EHRSHOT. Collectively, EHR-Ins, EHR-R1, and EHR-Bench have significantly advanced the development for more reliable and clinically relevant EHR analysis.