DREAM: Deep Research Evaluation with Agentic Metrics

TL;DR

DREAM introduces an agentic evaluation framework for research agents that improves assessment of factual accuracy and temporal validity through adaptive, tool-enabled metrics, surpassing existing static benchmarks.

Contribution

The paper presents DREAM, a novel agentic evaluation framework that enhances research report assessment by incorporating tool-use capabilities for better factual and temporal validation.

Findings

DREAM is more sensitive to factual decay than existing benchmarks.

The framework enables temporally aware and grounded evaluation.

DREAM offers a scalable, reference-free assessment method.

Abstract

Deep Research Agents generate analyst-grade reports, yet evaluating them remains challenging due to the absence of a single ground truth and the multidimensional nature of research quality. Recent benchmarks propose distinct methodologies, yet they suffer from the Mirage of Synthesis, where strong surface-level fluency and citation alignment can obscure underlying factual and reasoning defects. We characterize this gap by introducing a taxonomy across four verticals that exposes a critical capability mismatch: static evaluators inherently lack the tool-use capabilities required to assess temporal validity and factual correctness. To address this, we propose DREAM (Deep Research Evaluation with Agentic Metrics), a framework that instantiates the principle of capability parity by making evaluation itself agentic. DREAM structures assessment through an evaluation protocol combining query-agnostic metrics with adaptive metrics generated by a tool-calling agent, enabling temporally aware coverage, grounded verification, and systematic reasoning probes. Controlled evaluations demonstrate DREAM is significantly more sensitive to factual and temporal decay than existing benchmarks, offering a scalable, reference-free evaluation paradigm.