The Untold Impact of Learning Approaches on Software Fault-Proneness Predictions

TL;DR

This study investigates how different learning approaches significantly influence software fault-proneness prediction performance, revealing that approach choice impacts results and should be explicitly considered in research and practice.

Contribution

It is the first comprehensive analysis of how learning approaches affect fault prediction, demonstrating the importance of approach selection and class imbalance handling.

Findings

useAllPredictAll approach outperforms usePrePredictPost in classification accuracy

Class imbalance explains performance differences between approaches

Addressing class imbalance equalizes the performance of different approaches

Abstract

Software fault-proneness prediction is an active research area, with many factors affecting prediction performance extensively studied. However, the impact of the learning approach (i.e., the specifics of the data used for training and the target variable being predicted) on the prediction performance has not been studied, except for one initial work. This paper explores the effects of two learning approaches, useAllPredictAll and usePrePredictPost, on the performance of software fault-proneness prediction, both within-release and across-releases. The empirical results are based on data extracted from 64 releases of twelve open-source projects. Results show that the learning approach has a substantial, and typically unacknowledged, impact on the classification performance. Specifically, using useAllPredictAll leads to significantly better performance than using usePrePredictPost learning approach, both within-release and across-releases. Furthermore, this paper uncovers that, for within-release predictions, this difference in classification performance is due to different levels of class imbalance in the two learning approaches. When class imbalance is addressed, the performance difference between the learning approaches is eliminated. Our findings imply that the learning approach should always be explicitly identified and its impact on software fault-proneness prediction considered. The paper concludes with a discussion of potential consequences of our results for both research and practice.