DRACO: a Cross-Domain Benchmark for Deep Research Accuracy, Completeness, and Objectivity

TL;DR

DRACO is a comprehensive benchmark designed to evaluate deep research systems across multiple domains, focusing on accuracy, completeness, objectivity, and citation quality using real-world, anonymized tasks.

Contribution

It introduces a large-scale, multi-domain benchmark with real-world tasks and detailed evaluation rubrics for assessing deep research system performance.

Findings

Benchmark covers 10 domains and 40 countries.

Tasks are anonymized, complex, and open-ended.

Evaluation includes factual accuracy, analysis depth, presentation, and citations.

Abstract

We present DRACO (Deep Research Accuracy, Completeness, and Objectivity), a benchmark of complex deep research tasks. These tasks, which span 10 domains and draw on information sources from 40 countries, originate from anonymized real-world usage patterns within a large-scale deep research system. Tasks are sampled from a de-identified dataset of Perplexity Deep Research requests, then filtered and augmented to ensure that the tasks are anonymized, open-ended and complex, objectively evaluable, and representative of the broad scope of real-world deep research use cases. Outputs are graded against task-specific rubrics along four dimensions: factual accuracy (accuracy), breadth and depth of analysis (including completeness), presentation quality (including objectivity), and citation quality. DRACO is publicly available at https://hf.co/datasets/perplexity-ai/draco.