Learning Query-Specific Rubrics from Human Preferences for DeepResearch Report Generation

TL;DR

This paper introduces a method to generate query-specific rubrics aligned with human preferences for evaluating DeepResearch reports, improving report quality and evaluation granularity.

Contribution

It proposes a novel pipeline combining reinforcement learning and multi-agent workflows to create more effective, human-aligned rubrics for report generation.

Findings

Rubric generators outperform existing strategies in discriminative power.

Integrated system achieves state-of-the-art results on DeepResearch Bench.

System matches performance of leading closed-source models.

Abstract

Nowadays, training and evaluating DeepResearch-generated reports remain challenging due to the lack of verifiable reward signals. Accordingly, rubric-based evaluation has become a common practice. However, existing approaches either rely on coarse, pre-defined rubrics that lack sufficient granularity, or depend on manually constructed query-specific rubrics that are costly and difficult to scale. In this paper, we propose a pipeline to train human-preference-aligned query-specific rubric generators tailored for DeepResearch report generation. We first construct a dataset of DeepResearch-style queries annotated with human preferences over paired reports, and train rubric generators via reinforcement learning with a hybrid reward combining human preference supervision and LLM-based rubric evaluation. To better handle long-horizon reasoning, we further introduce a Multi-agent Markov-state (MaMs) workflow for report generation. We empirically show that our proposed rubric generators deliver more discriminative and better human-aligned supervision than existing rubric design strategies. Moreover, when integrated into the MaMs training framework, DeepResearch systems equipped with our rubric generators consistently outperform all open-source baselines on the DeepResearch Bench and achieve performance comparable to that of leading closed-source models.