ReviewRanker: A Semi-Supervised Learning Based Approach for Code Review Quality Estimation

TL;DR

ReviewRanker is a semi-supervised learning system designed to automatically estimate the quality of code reviews, reducing human bias and effort, thereby enhancing the efficiency and effectiveness of the code review process.

Contribution

It introduces a semi-supervised approach that uses simple developer-provided labels to predict review quality scores with minimal effort and bias.

Findings

Reduces human bias in review quality assessment

Requires minimal labeling effort from developers

Potential to streamline the code review cycle

Abstract

Code review is considered a key process in the software industry for minimizing bugs and improving code quality. Inspection of review process effectiveness and continuous improvement can boost development productivity. Such inspection is a time-consuming and human-bias-prone task. We propose a semi-supervised learning based system ReviewRanker which is aimed at assigning each code review a confidence score which is expected to resonate with the quality of the review. Our proposed method is trained based on simple and and well defined labels provided by developers. The labeling task requires little to no effort from the developers and has an indirect relation to the end goal (assignment of review confidence score). ReviewRanker is expected to improve industry-wide code review quality inspection through reducing human bias and effort required for such task. The system has the potential of minimizing the back-and-forth cycle existing in the development and review process. Usable code and dataset for this research can be found at: https://github.com/saifarnab/code_review