IRJIT: A Simple, Online, Information Retrieval Approach for Just-In-Time Software Defect Prediction

TL;DR

IRJIT is an online, explainable, information retrieval-based method for just-in-time software defect prediction that is faster and maintains competitive accuracy compared to traditional machine learning models.

Contribution

The paper introduces IRJIT, a novel online and explainable approach for defect prediction using information retrieval, addressing the limitations of existing complex ML-based methods.

Findings

IRJIT is up to 112 times faster than state-of-the-art approaches.

IRJIT provides explainability at commit and line levels.

IRJIT achieves comparable prediction performance to existing methods.

Abstract

Just-in-Time software defect prediction (JIT-SDP) prevents the introduction of defects into the software by identifying them at commit check-in time. Current software defect prediction approaches rely on manually crafted features such as change metrics and involve expensive to train machine learning or deep learning models. These models typically involve extensive training processes that may require significant computational resources and time. These characteristics can pose challenges when attempting to update the models in real-time as new examples become available, potentially impacting their suitability for fast online defect prediction. Furthermore, the reliance on a complex underlying model makes these approaches often less explainable, which means the developers cannot understand the reasons behind models' predictions. An approach that is not explainable might not be adopted in real-life development environments because of developers' lack of trust in its results. To address these limitations, we propose an approach called IRJIT that employs information retrieval on source code and labels new commits as buggy or clean based on their similarity to past buggy or clean commits. IRJIT approach is online and explainable as it can learn from new data without expensive retraining, and developers can see the documents that support a prediction, providing additional context. By evaluating 10 open-source datasets in a within project setting, we show that our approach is up to 112 times faster than the state-of-the-art ML and DL approaches, offers explainability at the commit and line level, and has comparable performance to the state-of-the-art.