Bug or not bug? That is the question

TL;DR

This paper presents a machine learning approach combining NLP and neural networks, optimized via genetic algorithms, to accurately classify issues as bugs or non-bugs, improving issue management accuracy.

Contribution

It introduces a novel binary classification method for bug detection that outperforms existing solutions on standard benchmarks.

Findings

Significant F1 score improvements over existing methods

Effective hyper-parameter optimization using genetic algorithms

Enhanced accuracy in bug versus non-bug issue classification

Abstract

Nowadays, development teams often rely on tools such as Jira or Bugzilla to manage backlogs of issues to be solved to develop or maintain software. Although they relate to many different concerns (e.g., bug fixing, new feature development, architecture refactoring), few means are proposed to identify and classify these different kinds of issues, except for non mandatory labels that can be manually associated to them. This may lead to a lack of issue classification or to issue misclassification that may impact automatic issue management (planning, assignment) or issue-derived metrics. Automatic issue classification thus is a relevant topic for assisting backlog management. This paper proposes a binary classification solution for discriminating bug from non bug issues. This solution combines natural language processing (TF-IDF) and classification (multi-layer perceptron) techniques, selected after comparing commonly used solutions to classify issues. Moreover, hyper-parameters of the neural network are optimized using a genetic algorithm. The obtained results, as compared to existing works on a commonly used benchmark, show significant improvements on the F1 measure for all datasets.