AgentEHR: Advancing Autonomous Clinical Decision-Making via Retrospective Summarization

TL;DR

AgentEHR introduces a new benchmark for autonomous clinical decision-making with a novel retrospective summarization framework, RetroSum, which enhances reasoning continuity and experience retrieval, leading to significant performance improvements.

Contribution

The paper presents RetroSum, a novel framework combining retrospective summarization and evolving experience strategies to improve autonomous EHR navigation.

Findings

RetroSum outperforms baselines by up to 29.16% in accuracy.

It reduces interaction errors by up to 92.3%.

The benchmark challenges complex clinical decision tasks.

Abstract

Large Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records~(EHRs) navigation remains constrained by a reliance on curated inputs and simplified retrieval tasks. To bridge the gap between idealized experimental settings and realistic clinical environments, we present AgentEHR. This benchmark challenges agents to execute complex decision-making tasks, such as diagnosis and treatment planning, requiring long-range interactive reasoning directly within raw and high-noise databases. In tackling these tasks, we identify that existing summarization methods inevitably suffer from critical information loss and fractured reasoning continuity. To address this, we propose RetroSum, a novel framework that unifies a retrospective summarization mechanism with an evolving experience strategy. By dynamically re-evaluating interaction history, the retrospective mechanism prevents long-context information loss and ensures unbroken logical coherence. Additionally, the evolving strategy bridges the domain gap by retrieving accumulated experience from a memory bank. Extensive empirical evaluations demonstrate that RetroSum achieves performance gains of up to 29.16% over competitive baselines, while significantly decreasing total interaction errors by up to 92.3%.