On The Effectiveness of One-Class Support Vector Machine in Different Defect Prediction Scenarios

TL;DR

This study evaluates the effectiveness of One-Class Support Vector Machine (OCSVM) for defect prediction across different granularities and scenarios, finding it performs better in heterogeneous data settings but not as well as two-class classifiers.

Contribution

The paper extends previous work by empirically testing OCSVM in cross-version and cross-project defect prediction scenarios, highlighting its strengths and limitations.

Findings

OCSVM outperforms SVM and other two-class classifiers in certain scenarios.

OCSVM performs better in cross-version and cross-project settings than within-project.

Two-class Random Forest generally outperforms OCSVM in defect prediction.

Abstract

Defect prediction aims at identifying software components that are likely to cause faults before a software is made available to the end-user. To date, this task has been modeled as a two-class classification problem, however its nature also allows it to be formulated as a one-class classification task. Previous studies show that One-Class Support Vector Machine (OCSVM) can outperform two-class classifiers for within-project defect prediction, however it is not effective when employed at a finer granularity (i.e., commit-level defect prediction). In this paper, we further investigate whether learning from one class only is sufficient to produce effective defect prediction model in two other different scenarios (i.e., granularity), namely cross-version and cross-project defect prediction models, as well as replicate the previous work at within-project granularity for completeness. Our empirical results confirm that OCSVM performance remain low at different granularity levels, that is, it is outperformed by the two-class Random Forest (RF) classifier for both cross-version and cross-project defect prediction. While, we cannot conclude that OCSVM is the best classifier, our results still show interesting findings. While OCSVM does not outperform RF, it still achieves performance superior to its two-class counterpart (i.e., SVM) as well as other two-class classifiers studied herein. We also observe that OCSVM is more suitable for both cross-version and cross-project defect prediction, rather than for within-project defect prediction, thus suggesting it performs better with heterogeneous data. We encourage further research on one-class classifiers for defect prediction as these techniques may serve as an alternative when data about defective modules is scarce or not available.