Feature Importance in the Context of Traditional and Just-In-Time Software Defect Prediction Models

TL;DR

This paper compares traditional and Just-In-Time software defect prediction models, highlighting feature importance using deep learning and explainability techniques, and reports high accuracy and AUC scores for both approaches.

Contribution

It introduces a comparative analysis of feature importance in traditional and JIT defect prediction models using deep learning and explainability methods.

Findings

Deep learning achieved 80-86% accuracy in defect prediction.

JIT models had higher AUC scores than traditional models.

Feature importance was effectively identified using SHAP and integrated gradients.

Abstract

Software defect prediction models can assist software testing initiatives by prioritizing testing error-prone modules. In recent years, in addition to the traditional defect prediction model approach of predicting defects from class, modules, etc., Just-In-Time defect prediction research, which focuses on the change history of software products is getting prominent. For building these defect prediction models, it is important to understand which features are primary contributors to these classifiers. This study considered developing defect prediction models incorporating the traditional and the Just-In-Time approaches from the publicly available dataset of the Apache Camel project. A multi-layer deep learning algorithm was applied to these datasets in comparison with machine learning algorithms. The deep learning algorithm achieved accuracies of 80% and 86%, with the area under receiving operator curve (AUC) scores of 66% and 78% for traditional and Just-In-Time defect prediction, respectively. Finally, the feature importance of these models was identified using a model-specific integrated gradient method and a model-agnostic Shapley Additive Explanation (SHAP) technique.