CodeFlowLM: Incremental Just-In-Time Defect Prediction with Pretrained Language Models and Exploratory Insights into Defect Localization

TL;DR

CodeFlowLM is an incremental learning framework using pretrained language models for just-in-time defect prediction, demonstrating significant improvements in adaptability and robustness in dynamic software environments.

Contribution

The paper introduces CodeFlowLM, a novel incremental fine-tuning approach with pretrained models for JIT-SDP, and provides exploratory insights into defect localization using large language models.

Findings

CodeFlowLM achieves up to 68% G-Mean gains in JIT-SDP.

GPT-5 performs comparably to attention-based models in defect localization.

Analysis reveals false positives mainly stem from conservative bias and limited context.

Abstract

This work introduces CodeFlowLM, an incremental learning framework for Just-In-Time Software Defect Prediction (JIT-SDP) that leverages pre-trained language models (PLMs). Unlike traditional online learners, CodeFlowLM employs continual fine-tuning to address concept drift, class imbalance, and verification latency without retraining from scratch. We evaluated encoder-only and encoder-decoder PLMs (notably CodeT5+ and UniXCoder) in JIT-SDP scenarios within and between projects, comparing them with the incremental baseline BORB. The results show that CodeFlowLM achieves up to 68% G-Mean gains, confirming its superior adaptability and robustness in evolving software environments. We further extend the analysis to Just-in-Time Defect Localization (JIT-DL), benchmarking Large Language Models (LLMs) such as GPT-5, Claude Sonnet 4.5, and Gemini 2.5 Pro against attention-based models. GPT-5 delivers comparable performance for Recall@20% and Effort@20% with higher stability, although attention-based methods retain an advantage in fine-grained ranking metrics (Top-k, IFA). A qualitative error analysis reveals that most false positives arise from (1) human-like conservative bias, (2) insufficient contextual information in diff-based prompts, and (3) potential dataset mislabeling in JIT-Defects4J. These findings highlight both the promise and the current limitations of LLM reasoning in defect localization. False negatives occur in smaller proportions. Overall, CodeFlowLM significantly advances the state of the art in incremental JIT-SDP, demonstrating superior adaptability and robustness in evolving software environments. Furthermore, our exploratory analysis of LLMs in JIT-DL not only benchmarks their performance against established attention-based models but also provides critical insights into the current limitations of prompt-based defect reasoning.