DRPG (Decompose, Retrieve, Plan, Generate): An Agentic Framework for Academic Rebuttal

TL;DR

DRPG is an innovative agentic framework that enhances automated academic rebuttal generation by decomposing reviews, retrieving evidence, planning strategies, and generating targeted responses, outperforming existing methods.

Contribution

The paper introduces DRPG, a novel four-step framework that significantly improves automated rebuttal quality and explainability in academic peer review processes.

Findings

DRPG's planner achieves over 98% accuracy in identifying rebuttal directions.

Outperforms existing rebuttal pipelines on top-tier conference data.

Achieves performance beyond average human levels with an 8B model.

Abstract

Despite the growing adoption of large language models (LLMs) in scientific research workflows, automated support for academic rebuttal, a crucial step in academic communication and peer review, remains largely underexplored. Existing approaches typically rely on off-the-shelf LLMs or simple pipelines, which struggle with long-context understanding and often fail to produce targeted and persuasive responses. In this paper, we propose DRPG, an agentic framework for automatic academic rebuttal generation that operates through four steps: Decompose reviews into atomic concerns, Retrieve relevant evidence from the paper, Plan rebuttal strategies, and Generate responses accordingly. Notably, the Planner in DRPG reaches over 98% accuracy in identifying the most feasible rebuttal direction. Experiments on data from top-tier conferences demonstrate that DRPG significantly outperforms existing rebuttal pipelines and achieves performance beyond the average human level using only an 8B model. Our analysis further demonstrates the effectiveness of the planner design and its value in providing multi-perspective and explainable suggestions. We also showed that DRPG works well in a more complex multi-round setting. These results highlight the effectiveness of DRPG and its potential to provide high-quality rebuttal content and support the scaling of academic discussions. Codes for this work are available at https://github.com/ulab-uiuc/DRPG-RebuttalAgent.