Issue-Oriented Agent-Based Framework for Automated Review Comment Generation

TL;DR

RevAgent is an innovative agent-based framework that improves automated code review comment generation by decomposing the task into specialized stages, significantly outperforming existing models in accuracy and efficiency.

Contribution

The paper introduces RevAgent, a novel multi-stage, issue-oriented framework that enhances comment relevance and accuracy in automated code reviews through specialized agents and fine-tuning.

Findings

RevAgent outperforms state-of-the-art baselines in BLEU, ROUGE-L, METEOR, and SBERT metrics.

It achieves higher accuracy in issue-category identification, especially in complex scenarios.

Human evaluation confirms RevAgent's effectiveness in generating practical, context-aware review comments.

Abstract

Code review (CR) is a crucial practice for ensuring software quality. Various automated review comment generation techniques have been proposed to streamline the labor-intensive process. However, existing approaches heavily rely on a single model to identify various issues within the code, limiting the model's ability to handle the diverse, issue-specific nature of code changes and leading to non-informative comments, especially in complex scenarios such as bug fixes. To address these limitations, we propose RevAgent, a novel agent-based issue-oriented framework, decomposes the task into three stages: (1) Generation Stage, where five category-specific commentator agents analyze code changes from distinct issue perspectives and generate candidate comments; (2) Discrimination Stage, where a critic agent selects the most appropriate issue-comment pair; and (3) Training Stage, where all agents are fine-tuned on curated, category-specific data to enhance task specialization. Evaluation results show that RevAgent significantly outperforms state-of-the-art PLM- and LLM-based baselines, with improvements of 12.90\%, 10.87\%, 6.32\%, and 8.57\% on BLEU, ROUGE-L, METEOR, and SBERT, respectively. It also achieves relatively higher accuracy in issue-category identification, particularly for challenging scenarios. Human evaluations further validate the practicality of RevAgent in generating accurate, readable, and context-aware review comments. Moreover, RevAgent delivers a favorable trade-off between performance and efficiency.