Moving from Cross-Project Defect Prediction to Heterogeneous Defect Prediction: A Partial Replication Study

TL;DR

This study systematically evaluates the feasibility of heterogeneous defect prediction (HDP) models that transfer knowledge across projects with different metrics, comparing their performance to traditional cross-project defect prediction and providing practical guidelines.

Contribution

It extends prior research by replicating and validating HDP results, proposing a new ensemble voting approach, and analyzing the conditions affecting HDP's feasibility.

Findings

HDP performance is comparable to traditional methods in some cases.

HDP sensitivity to parameter selection limits its applicability.

Guidelines are provided for effective transfer learning in defect prediction.

Abstract

Software defect prediction heavily relies on the metrics collected from software projects. Earlier studies often used machine learning techniques to build, validate, and improve bug prediction models using either a set of metrics collected within a project or across different projects. However, techniques applied and conclusions derived by those models are restricted by how identical those metrics are. Knowledge coming from those models will not be extensible to a target project if no sufficient overlapping metrics have been collected in the source projects. To explore the feasibility of transferring knowledge across projects without common labeled metrics, we systematically integrated Heterogeneous Defect Prediction (HDP) by replicating and validating the obtained results. Our main goal is to extend prior research and explore the feasibility of HDP and finally to compare its performance with that of its predecessor, Cross-Project Defect Prediction. We construct an HDP model on different publicly available datasets. Moreover, we propose a new ensemble voting approach in the HDP context to utilize the predictive power of multiple available datasets. The result of our experiment is comparable to that of the original study. However, we also explored the feasibility of HDP in real cases. Our results shed light on the infeasibility of many cases for the HDP algorithm due to its sensitivity to the parameter selection. In general, our analysis gives a deep insight into why and how to perform transfer learning from one domain to another, and in particular, provides a set of guidelines to help researchers and practitioners to disseminate knowledge to the defect prediction domain.