How Do Developers Use Code Suggestions in Pull Request Reviews?

TL;DR

This study investigates how developers utilize GitHub's code suggestion feature in pull request reviews, revealing its types, usage patterns, and impact on merge rates, resolution time, and social dynamics.

Contribution

It provides an empirical analysis of suggestion types, their effects on pull request outcomes, and insights into social factors influencing their usage and acceptance.

Findings

Suggestions increase merge rates.

Suggestions raise resolution time.

Suggestions do not reduce code complexity.

Abstract

GitHub introduced the suggestion feature to enable reviewers to explicitly suggest code modifications in pull requests. These suggestions make the reviewers' feedback more actionable for the submitters and represent a valuable knowledge for newcomers. Still, little is known about how code review suggestions are used by developers, what impact they have on pull requests, and how they are influenced by social coding dynamics. To bridge this knowledge gap, we conducted an empirical study on pull requests from 46 engineered GitHub projects, in which developers used code review suggestions. We applied an open coding approach to uncover the types of suggestions and their usage frequency. We also mined pull request characteristics and assessed the impact of using suggestions on merge rate, resolution time, and code complexity. Furthermore, we conducted a survey with contributors of the studied projects to gain insights about the influence of social factors on the usage and acceptance of code review suggestions. We were able to uncover four suggestion types: code style suggestions, improvements, fixes, and documentation with improvements being the most frequent. We found that the use of suggestions positively affects the merge rate of pull requests but significantly increases resolution time without leading to a decrease in code complexity. Our survey results show that suggestions are more likely to be used by reviewers when the submitter is a newcomer. The results also show that developers mostly search suggestions when tracking rationale or looking for code examples. Our work provides insights on the usage of code suggestions and their potential as a knowledge sharing tool.