Exploring the Advances in Identifying Useful Code Review Comments

TL;DR

This paper reviews the evolution of research on identifying useful code review comments, highlighting methods, datasets, perceptions, and machine learning approaches, and discusses future challenges in automating usefulness prediction.

Contribution

It provides a comprehensive overview of existing research on useful code review comments, including datasets, perception studies, and machine learning techniques, and outlines open problems.

Findings

Machine learning classifiers can predict comment usefulness.

Perceptions of usefulness vary among developers.

Open challenges remain in automating usefulness recognition.

Abstract

Effective peer code review in collaborative software development necessitates useful reviewer comments and supportive automated tools. Code review comments are a central component of the Modern Code Review process in the industry and open-source development. Therefore, it is important to ensure these comments serve their purposes. This paper reflects the evolution of research on the usefulness of code review comments. It examines papers that define the usefulness of code review comments, mine and annotate datasets, study developers' perceptions, analyze factors from different aspects, and use machine learning classifiers to automatically predict the usefulness of code review comments. Finally, it discusses the open problems and challenges in recognizing useful code review comments for future research.