Synthetic Minority Over-sampling TEchnique(SMOTE) for Predicting Software Build Outcomes

TL;DR

This paper applies a data stream approach combined with SMOTE oversampling to predict software build outcomes using decision trees, demonstrating improved accuracy over time with a focus on the significance of specific metrics.

Contribution

It introduces the use of SMOTE in a data stream context for predicting software build success, highlighting the evolution of model accuracy over incremental data.

Findings

Classification accuracy reaches 80% after 900 instances.

Few metrics significantly influence build outcome predictions.

Model accuracy improves steadily with more data.

Abstract

In this research we use a data stream approach to mining data and construct Decision Tree models that predict software build outcomes in terms of software metrics that are derived from source code used in the software construction process. The rationale for using the data stream approach was to track the evolution of the prediction model over time as builds are incrementally constructed from previous versions either to remedy errors or to enhance functionality. As the volume of data available for mining from the software repository that we used was limited, we synthesized new data instances through the application of the SMOTE oversampling algorithm. The results indicate that a small number of the available metrics have significance for prediction software build outcomes. It is observed that classification accuracy steadily improves after approximately 900 instances of builds have been fed to the classifier. At the end of the data streaming process classification accuracies of 80% were achieved, though some bias arises due to the distribution of data across the two classes over time.