Descriptions of issues and comments for predicting issue success in software projects

TL;DR

This study investigates how textual descriptions and comments in Issue Tracking Systems can predict the success of software issues, achieving over 85% accuracy across various issue types and highlighting the importance of development-related language.

Contribution

It introduces a large-scale analysis of textual data from Issue Tracking Systems to predict issue success, demonstrating the effectiveness of machine learning classifiers in this context.

Findings

Textual descriptions and comments are highly predictive of issue success.

Development-related words significantly influence prediction accuracy.

Issue success predictions vary over time, indicating dynamic factors.

Abstract

Software development tasks must be performed successfully to achieve software quality and customer satisfaction. Knowing whether software tasks are likely to fail is essential to ensure the success of software projects. Issue Tracking Systems store information of software tasks (issues) and comments, which can be useful to predict issue success; however; almost no research on this topic exists. This work studies the usefulness of textual descriptions of issues and comments for predicting whether issues will be resolved successfully or not. Issues and comments of 588 software projects were extracted from four popular Issue Tracking Systems. Seven machine learning classifiers were trained on 30k issues and more than 120k comments, and more than 6000 experiments were performed to predict the success of three types of issues: bugs, improvements and new features. The results provided evidence that descriptions of issues and comments are useful for predicting issue success with more than 85% of accuracy and precision, and that the predictions of issue success vary over time. Words related to software development were particularly relevant for predicting issue success. Other communication aspects and their relationship to the success of software projects must be researched in detail using data from software tools.