The Integrity of Machine Learning Algorithms against Software Defect Prediction

TL;DR

This paper evaluates the effectiveness of the Online Sequential Extreme Learning Machine (OS-ELM) in predicting software defects, demonstrating its superior performance over traditional classifiers on imbalanced datasets.

Contribution

It introduces a comparison of OS-ELM with other classifiers using oversampled data, highlighting its faster training and better accuracy in defect prediction.

Findings

OS-ELM outperforms traditional classifiers in recall and balanced accuracy.

Oversampling with Cluster-based Over-Sampling with Noise Filtering improves results.

OS-ELM trains faster and converges to a global optimum.

Abstract

The increased computerization in recent years has resulted in the production of a variety of different software, however measures need to be taken to ensure that the produced software isn't defective. Many researchers have worked in this area and have developed different Machine Learning-based approaches that predict whether the software is defective or not. This issue can't be resolved simply by using different conventional classifiers because the dataset is highly imbalanced i.e the number of defective samples detected is extremely less as compared to the number of non-defective samples. Therefore, to address this issue, certain sophisticated methods are required. The different methods developed by the researchers can be broadly classified into Resampling based methods, Cost-sensitive learning-based methods, and Ensemble Learning. Among these methods. This report analyses the performance of the Online Sequential Extreme Learning Machine (OS-ELM) proposed by Liang et.al. against several classifiers such as Logistic Regression, Support Vector Machine, Random Forest, and Na\"ive Bayes after oversampling the data. OS-ELM trains faster than conventional deep neural networks and it always converges to the globally optimal solution. A comparison is performed on the original dataset as well as the over-sampled data set. The oversampling technique used is Cluster-based Over-Sampling with Noise Filtering. This technique is better than several state-of-the-art techniques for oversampling. The analysis is carried out on 3 projects KC1, PC4 and PC3 carried out by the NASA group. The metrics used for measurement are recall and balanced accuracy. The results are higher for OS-ELM as compared to other classifiers in both scenarios.