Measuring the effectiveness of code review comments in GitHub repositories: A machine learning approach

TL;DR

This study evaluates machine learning techniques for classifying GitHub code review comments by semantic meaning and sentiment polarity, aiming to improve developer understanding and error detection in open-source projects.

Contribution

It compares seven machine learning algorithms for classifying code review comments and identifies Linear SVC as the most accurate method for sentiment analysis.

Findings

Linear SVC achieves the highest accuracy among tested algorithms.

Manual labeling of 13,557 comments provides a substantial dataset.

The approach aids programmers in understanding and addressing code review comments.

Abstract

This paper illustrates an empirical study of the working efficiency of machine learning techniques in classifying code review text by semantic meaning. The code review comments from the source control repository in GitHub were extracted for development activity from the existing year for three open-source projects. Apart from that, programmers need to be aware of their code and point out their errors. In that case, it is a must to classify the sentiment polarity of the code review comments to avoid an error. We manually labelled 13557 code review comments generated by three open source projects in GitHub during the existing year. In order to recognize the sentiment polarity (or sentiment orientation) of code reviews, we use seven machine learning algorithms and compare those results to find the better ones. Among those Linear Support Vector Classifier(SVC) classifier technique achieves higher accuracy than others. This study will help programmers to make any solution based on code reviews by avoiding misconceptions.