Toward the Automatic Classification of Self-Affirmed Refactoring

TL;DR

This paper presents an automated approach to classify self-affirmed refactoring commits in software development, improving accuracy over previous manual methods and uncovering additional refactoring patterns.

Contribution

It introduces a two-step machine learning model combining N-Gram TF-IDF and classifiers to automatically categorize refactoring commits based on quality improvement categories.

Findings

Model achieves up to 90% F-measure in classification accuracy.

Outperforms pattern-based and random classifiers.

Discovers 40 additional relevant SAR patterns.

Abstract

The concept of Self-Affirmed Refactoring (SAR) was introduced to explore how developers document their refactoring activities in commit messages, i.e., developers' explicit documentation of refactoring operations intentionally introduced during a code change. In our previous study, we have manually identified refactoring patterns and defined three main common quality improvement categories, including internal quality attributes, external quality attributes, and code smells, by only considering refactoring-related commits. However, this approach heavily depends on the manual inspection of commit messages. In this paper, we propose a two-step approach to first identify whether a commit describes developer-related refactoring events, then to classify it according to the refactoring common quality improvement categories. Specifically, we combine the N-Gram TF-IDF feature selection with binary and multiclass classifiers to build a new model to automate the classification of refactorings based on their quality improvement categories. We challenge our model using a total of 2,867 commit messages extracted from well-engineered open-source Java projects. Our findings show that (1) our model is able to accurately classify SAR commits, outperforming the pattern-based and random classifier approaches, and allowing the discovery of 40 more relevant SAR patterns, and (2) our model reaches an F-measure of up to 90% even with a relatively small training dataset.