Code Review as Decision-Making -- Building a Cognitive Model from the Questions Asked During Code Review

TL;DR

This paper develops a cognitive model of code review based on decision-making processes, revealing how developers navigate review phases and make decisions, with implications for improving review tools and practices.

Contribution

It introduces the CRDM model, linking code review to recognition-primed decision-making, based on ethnographic analysis of real review sessions.

Findings

Code review involves two main phases: orientation and analysis.

Developers make multiple decisions, such as commenting, testing, and verifying results.

The cognitive process resembles recognition-primed decision-making, informing better tool design.

Abstract

Code review is a well-established and valued practice in the software engineering community contributing to both code quality and interpersonal benefits. However, there are challenges in both tools and processes that give rise to misalignments and frustrations. Recent research seeks to address this by automating code review entirely, but we believe that this risks losing the majority of the interpersonal benefits such as knowledge transfer and shared ownership. We believe that by better understanding the cognitive processes involved in code review, it would be possible to improve tool support, with out without AI, and make code review both more efficient, more enjoyable, while increasing or maintaining all of its benefits. In this paper, we conduct an ethnographic think-aloud study involving 10 participants and 34 code reviews. We build a cognitive model of code review bottom up through thematic, statistical, temporal, and sequential analysis of the transcribed material. Through the data, the similarities between the cognitive process in code review and decision-making processes, especially recognition-primed decision-making, become apparent. The result is the Code Review as Decision-Making (CRDM) model that shows how the developers move through two phases during the code review; first an orientation phase to establish context and rationale and then an analytical phase to understand, assess, and plan the rest of the review. Throughout the process several decisions must be taken, on writing comments, finding more information, voting, running the code locally, verifying continuous integration results, etc. Analysis software and process-coded data publicly available at: https://doi.org/10.5281/zenodo.15758266