Performance Comparison of Binary Machine Learning Classifiers in Identifying Code Comment Types: An Exploratory Study

TL;DR

This study compares various binary machine learning classifiers for identifying different categories of code comments across three programming languages, highlighting Linear SVC as the most effective with an average F1 score of 0.5474.

Contribution

It introduces a comprehensive comparison of 19 classifiers for categorizing code comments, demonstrating the effectiveness of Linear SVC in this task.

Findings

Linear SVC achieved the highest average F1 score of 0.5474.

Performance varies across classifiers and comment categories.

The study covers three different programming languages.

Abstract

Code comments are vital to source code as they help developers with program comprehension tasks. Written in natural language (usually English), code comments convey a variety of different information, which are grouped into specific categories. In this study, we construct 19 binary machine learning classifiers for code comment categories that belong to three different programming languages. We present a comparison of performance scores for different types of machine learning classifiers and show that the Linear SVC classifier has the highest average F1 score of 0.5474.