Adapting Neural Text Classification for Improved Software Categorization

TL;DR

This paper enhances neural text classification methods for software categorization, addressing the challenges of applying NLP techniques directly to source code and comments, and demonstrates improved accuracy through tailored adaptations.

Contribution

The paper introduces specific adaptations to neural classification algorithms that significantly improve software categorization accuracy over existing methods.

Findings

Achieved higher classification accuracy than previous techniques.

Neural adaptations perform well on both Debian programs and annotated C/C++ libraries.

Proposed method outperforms standard neural text classifiers on software data.

Abstract

Software Categorization is the task of organizing software into groups that broadly describe the behavior of the software, such as "editors" or "science." Categorization plays an important role in several maintenance tasks, such as repository navigation and feature elicitation. Current approaches attempt to cast the problem as text classification, to make use of the rich body of literature from the NLP domain. However, as we will show in this paper, text classification algorithms are generally not applicable off-the-shelf to source code; we found that they work well when high-level project descriptions are available, but suffer very large performance penalties when classifying source code and comments only. We propose a set of adaptations to a state-of-the-art neural classification algorithm and perform two evaluations: one with reference data from Debian end-user programs, and one with a set of C/C++ libraries that we hired professional programmers to annotate. We show that our proposed approach achieves performance exceeding that of previous software classification techniques as well as a state-of-the-art neural text classification technique.