STACC: Code Comment Classification using SentenceTransformers

TL;DR

STACC introduces SentenceTransformers-based classifiers to automatically categorize code comments, significantly improving classification accuracy and aiding software maintenance and understanding.

Contribution

The paper presents a novel lightweight classification approach using SentenceTransformers, outperforming existing baselines on a standard dataset.

Findings

Achieved an average F1 score of 0.74, surpassing the baseline of 0.31.

Significant improvement of 139% over baseline performance.

Models and replication package are publicly available.

Abstract

Code comments are a key resource for information about software artefacts. Depending on the use case, only some types of comments are useful. Thus, automatic approaches to classify these comments have been proposed. In this work, we address this need by proposing, STACC, a set of SentenceTransformers-based binary classifiers. These lightweight classifiers are trained and tested on the NLBSE Code Comment Classification tool competition dataset, and surpass the baseline by a significant margin, achieving an average F1 score of 0.74 against the baseline of 0.31, which is an improvement of 139%. A replication package, as well as the models themselves, are publicly available.