Towards debiasing code review support

TL;DR

This paper investigates cognitive biases in code review, identifies harmful effects, and proposes prototype solutions like confirmation bias and decision fatigue mitigation, validated through usability testing and user feedback.

Contribution

It introduces prototype tools targeting cognitive biases in code review, validated with user-centered design methods, and demonstrates their acceptance and potential to improve review objectivity.

Findings

Prototypes help prevent biased behaviors in code review

Techniques are well accepted by reviewers

Some solutions can be integrated into existing tools

Abstract

Cognitive biases appear during code review. They significantly impact the creation of feedback and how it is interpreted by developers. These biases can lead to illogical reasoning and decision-making, violating one of the main hypotheses supporting code review: developers' accurate and objective code evaluation. This paper explores harmful cases caused by cognitive biases during code review and potential solutions to avoid such cases or mitigate their effects. In particular, we design several prototypes covering confirmation bias and decision fatigue. We rely on a developer-centered design approach by conducting usability tests and validating the prototype with a user experience questionnaire (UEQ) and participants' feedback. We show that some techniques could be implemented in existing code review tools as they are well accepted by reviewers and help prevent behavior detrimental to code review. This work provides a solid first approach to treating cognitive bias in code review.