A Systematic Literature Review and Taxonomy of Modern Code Review

TL;DR

This paper systematically reviews modern code review (MCR), categorizing existing research into foundational studies, proposals, and evaluations, highlighting current knowledge, gaps, and future research directions in software quality assurance.

Contribution

It provides a comprehensive taxonomy and structured overview of MCR research, identifying key categories, common approaches, and research gaps for future exploration.

Findings

Foundational studies dominate MCR research, focusing on motivations, challenges, and benefits.

Most proposals involve code reviewer recommendation and support tools.

Evaluations are mostly offline, lacking human subject involvement.

Abstract

Modern Code Review (MCR) is a widely known practice of software quality assurance. However, the existing body of knowledge of MCR is currently not understood as a whole. Objective: Our goal is to identify the state of the art on MCR, providing a structured overview and an in-depth analysis of the research done in this field. Method: We performed a systematic literature review, selecting publications from four digital libraries. Results: A total of 139 papers were selected and analyzed in three main categories. Foundational studies are those that analyze existing or collected data from the adoption of MCR. Proposals consist of techniques and tools to support MCR, while evaluations are studies to assess an approach or compare a set of them. Conclusion: The most represented category is foundational studies, mainly aiming to understand the motivations for adopting MCR, its challenges and benefits, and which influence factors lead to which MCR outcomes. The most common types of proposals are code reviewer recommender and support to code checking. Evaluations of MCR-supporting approaches have been done mostly offline, without involving human subjects. Five main research gaps have been identified, which point out directions for future work in the area.