RebuttalAgent: Strategic Persuasion in Academic Rebuttal via Theory of Mind

TL;DR

RebuttalAgent introduces a novel AI framework grounded in Theory of Mind for strategic academic rebuttal, utilizing a large dataset and reinforcement learning to improve persuasion effectiveness beyond surface-level linguistic approaches.

Contribution

This paper presents the first ToM-based framework for academic rebuttal, including a new dataset, a two-stage training process, and a specialized evaluator, advancing strategic persuasion in AI.

Findings

RebuttalAgent outperforms baseline models by 18.3% on automated metrics.

The framework surpasses proprietary models in automated and human evaluations.

Rebuttal-RM achieves higher scoring consistency with human preferences than GPT-4.

Abstract

Although artificial intelligence (AI) has become deeply integrated into various stages of the research workflow and achieved remarkable advancements, academic rebuttal remains a significant and underexplored challenge. This is because rebuttal is a complex process of strategic communication under severe information asymmetry rather than a simple technical debate. Consequently, current approaches struggle as they largely imitate surface-level linguistics, missing the essential element of perspective-taking required for effective persuasion. In this paper, we introduce RebuttalAgent, the first framework to ground academic rebuttal in Theory of Mind (ToM), operationalized through a ToM-Strategy-Response (TSR) framework that models reviewer mental state, formulates persuasion strategy, and generates evidence-based response. To train our agent, we construct RebuttalBench, a large-scale dataset synthesized via a novel critique-and-refine approach. Our training process consists of two stages, beginning with a supervised fine-tuning phase to equip the agent with ToM-based analysis and strategic planning capabilities, followed by a reinforcement learning phase leveraging the self-reward mechanism for scalable self-improvement. For reliable and efficient automated evaluation, we further develop Rebuttal-RM, a specialized evaluator trained on over 100K samples of multi-source rebuttal data, which achieves scoring consistency with human preferences surpassing powerful judge GPT-4.1. Extensive experiments show RebuttalAgent significantly outperforms the base model by an average of 18.3% on automated metrics, while also outperforming advanced proprietary models across both automated and human evaluations.