Predicting the Number of Reported Bugs in a Software Repository

TL;DR

This paper compares eight time series forecasting models, including LSTM, ARIMA, and Random Forest, to predict bug counts in a large open-source project, highlighting the strengths of each for short-term and long-term predictions.

Contribution

It evaluates the effectiveness of various forecasting models and the impact of exogenous variables on bug prediction accuracy in software repositories.

Findings

LSTM excels in long-term bug prediction.

Random Forest with exogenous variables performs best for short-term forecasts.

Incorporating release dates improves prediction accuracy.

Abstract

The bug growth pattern prediction is a complicated, unrelieved task, which needs considerable attention. Advance knowledge of the likely number of bugs discovered in the software system helps software developers in designating sufficient resources at a convenient time. The developers may also use such information to take necessary actions to increase the quality of the system and in turn customer satisfaction. In this study, we examine eight different time series forecasting models, including Long Short Term Memory Neural Networks (LSTM), auto-regressive integrated moving average (ARIMA), and Random Forest Regressor. Further, we assess the impact of exogenous variables such as software release dates by incorporating those into the prediction models. We analyze the quality of long-term prediction for each model based on different performance metrics. The assessment is conducted on Mozilla, which is a large open-source software application. The dataset is originally mined from Bugzilla and contains the number of bugs for the project between Jan 2010 and Dec 2019. Our numerical analysis provides insights on evaluating the trends in a bug repository. We observe that LSTM is effective when considering long-run predictions whereas Random Forest Regressor enriched by exogenous variables performs better for predicting the number of bugs in the short term.