Automatic Paper Reviewing with Heterogeneous Graph Reasoning over LLM-Simulated Reviewer-Author Debates

TL;DR

This paper introduces ReViewGraph, a framework that models reviewer-author debates as heterogeneous graphs with LLM-simulated multi-round interactions, improving paper review accuracy by capturing complex argumentative dynamics.

Contribution

It presents a novel graph reasoning approach over simulated debates, explicitly modeling diverse opinion relations to enhance review decision-making.

Findings

ReViewGraph outperforms baselines with 15.73% average improvement.

Explicit modeling of debate structures improves review quality.

Heterogeneous graph reasoning captures complex argumentative dynamics.

Abstract

Existing paper review methods often rely on superficial manuscript features or directly on large language models (LLMs), which are prone to hallucinations, biased scoring, and limited reasoning capabilities. Moreover, these methods often fail to capture the complex argumentative reasoning and negotiation dynamics inherent in reviewer-author interactions. To address these limitations, we propose ReViewGraph (Reviewer-Author Debates Graph Reasoner), a novel framework that performs heterogeneous graph reasoning over LLM-simulated multi-round reviewer-author debates. In our approach, reviewer-author exchanges are simulated through LLM-based multi-agent collaboration. Diverse opinion relations (e.g., acceptance, rejection, clarification, and compromise) are then explicitly extracted and encoded as typed edges within a heterogeneous interaction graph. By applying graph neural networks to reason over these structured debate graphs, ReViewGraph captures fine-grained argumentative dynamics and enables more informed review decisions. Extensive experiments on three datasets demonstrate that ReViewGraph outperforms strong baselines with an average relative improvement of 15.73%, underscoring the value of modeling detailed reviewer-author debate structures.