The Impact of Human Factors on the Participation Decision of Reviewers in Modern Code Review

TL;DR

This study investigates how human factors influence whether invited reviewers participate in modern code reviews, analyzing data from over 230,000 patches across multiple systems to identify key predictors of review participation.

Contribution

It identifies human factors such as reviewer experience and participation rate as significant predictors of review participation, providing insights for improving reviewer invitation strategies.

Findings

16%-66% of patches had non-responding invited reviewers

Human factors significantly predict review participation

Reviewer experience and participation rate are highly associated with participation decision

Abstract

Modern Code Review (MCR) plays a key role in software quality practices. In MCR process, a new patch (i.e., a set of code changes) is encouraged to be examined by reviewers in order to identify weaknesses in source code prior to an integration into main software repositories. To mitigate the risk of having future defects, prior work suggests that MCR should be performed with sufficient review participation. Indeed, recent work shows that a low number of participated reviewers is associated with poor software quality. However, there is a likely case that a new patch still suffers from poor review participation even though reviewers were invited. Hence, in this paper, we set out to investigate the factors that are associated with the participation decision of an invited reviewer. Through a case study of 230,090 patches spread across the Android, LibreOffice, OpenStack and Qt systems, we find that (1) 16%-66% of patches have at least one invited reviewer who did not respond to the review invitation; (2) human factors play an important role in predicting whether or not an invited reviewer will participate in a review; (3) a review participation rate of an invited reviewers and code authoring experience of an invited reviewer are highly associated with the participation decision of an invited reviewer. These results can help practitioners better understand about how human factors associate with the participation decision of reviewers and serve as guidelines for inviting reviewers, leading to a better inviting decision and a better reviewer participation.