ESL-Bench: An Event-Driven Synthetic Longitudinal Benchmark for Health Agents

TL;DR

ESL-Bench is a synthetic, event-driven benchmark designed to evaluate health agents over multi-year trajectories, enabling systematic assessment of reasoning across diverse health data sources.

Contribution

It introduces a comprehensive synthetic benchmark with structured ground truth for evaluating health agents' reasoning capabilities across multiple query types.

Findings

DB agents outperform memory RAG baselines by 10-20% on key query types.

Evaluation across five question dimensions shows varied method strengths and weaknesses.

Synthetic data allows for controlled, scalable assessment of health reasoning algorithms.

Abstract

Longitudinal health agents must reason across multi-source trajectories that combine continuous device streams, sparse clinical exams, and episodic life events - yet evaluating them is hard: real-world data cannot be released at scale, and temporally grounded attribution questions seldom admit definitive answers without structured ground truth. We present ESL-Bench, an event-driven synthesis framework and benchmark providing 100 synthetic users, each with a 1-5 year trajectory comprising a health profile, a multi-phase narrative plan, daily device measurements, periodic exam records, and an event log with explicit per-indicator impact parameters. Each indicator follows a baseline stochastic process driven by discrete events with sigmoid-onset, exponential-decay kernels under saturation and projection constraints; a hybrid pipeline delegates sparse semantic artifacts to LLM-based planning and dense indicator dynamics to algorithmic simulation with hard physiological bounds. Users are each paired with 100 evaluation queries across five dimensions - Lookup, Trend, Comparison, Anomaly, Explanation - stratified into Easy, Medium, and Hard tiers, with all ground-truth answers programmatically computable from the recorded event-indicator relationships. Evaluating 13 methods spanning LLMs with tools, DB-native agents, and memory-augmented RAG, we find that DB agents (48-58%) substantially outperform memory RAG baselines (30-38%), with the gap concentrated on Comparison and Explanation queries where multi-hop reasoning and evidence attribution are required.