AUGER: Automatically Generating Review Comments with Pre-training Models

TL;DR

AUGER leverages pre-training models to automatically generate useful code review comments, significantly improving efficiency and quality in software review processes by utilizing a large dataset and advanced NLP techniques.

Contribution

The paper introduces AUGER, a novel framework that uses pre-trained T5 models to generate review comments, outperforming baselines and providing a scalable solution for code review automation.

Findings

Outperforms baselines by 37.38% in ROUGE-L

29% of generated comments are deemed useful

Inference time is approximately 20 seconds

Abstract

Code review is one of the best practices as a powerful safeguard for software quality. In practice, senior or highly skilled reviewers inspect source code and provide constructive comments, considering what authors may ignore, for example, some special cases. The collaborative validation between contributors results in code being highly qualified and less chance of bugs. However, since personal knowledge is limited and varies, the efficiency and effectiveness of code review practice are worthy of further improvement. In fact, it still takes a colossal and time-consuming effort to deliver useful review comments. This paper explores a synergy of multiple practical review comments to enhance code review and proposes AUGER (AUtomatically GEnerating Review comments): a review comments generator with pre-training models. We first collect empirical review data from 11 notable Java projects and construct a dataset of 10,882 code changes. By leveraging Text-to-Text Transfer Transformer (T5) models, the framework synthesizes valuable knowledge in the training stage and effectively outperforms baselines by 37.38% in ROUGE-L. 29% of our automatic review comments are considered useful according to prior studies. The inference generates just in 20 seconds and is also open to training further. Moreover, the performance also gets improved when thoroughly analyzed in case study.