AgentSZZ: Teaching the LLM Agent to Play Detective with Bug-Inducing Commits

TL;DR

AgentSZZ introduces an LLM-driven agent framework with interactive reasoning and domain tools to improve bug-inducing commit identification, outperforming existing methods especially in complex scenarios.

Contribution

It presents a novel agent-based approach that integrates tools and domain knowledge with iterative reasoning to enhance bug traceability in software repositories.

Findings

AgentSZZ achieves up to 27.2% higher F1-score than prior LLM-based approaches.

Recall improvements of up to 300% in challenging cross-file and ghost commit scenarios.

Structured compression reduces token usage by over 30% without significant accuracy loss.

Abstract

The SZZ algorithm is the dominant technique for identifying bug-inducing commits and underpins many software engineering tasks, such as defect prediction and vulnerability analysis. Despite numerous variants, including recent LLM-based approaches, performance remains limited on developer-annotated datasets (e.g., recall of 0.552 on the Linux kernel). A key limitation is the reliance on git blame, which traces line-level changes within the same file, failing in common scenarios such as ghost and cross-file cases-making nearly one-quarter of bug-inducing commits inherently untraceable. Moreover, current approaches follow fixed pipelines that restrict iterative reasoning and exploration, unlike developers who investigate bugs through an interactive, multi-tool process. To address these challenges, we propose AgentSZZ, an agent-based framework that leverages LLM-driven agents to explore repositories and identify bug-inducing commits. Unlike prior methods, AgentSZZ integrates task-specific tools, domain knowledge, and a ReAct-style loop to enable adaptive and causal tracing of bugs. A structured compression module further improves efficiency by reducing redundant context while preserving key evidence. Extensive experiments on three widely used datasets show that AgentSZZ consistently outperforms state-of-the-art SZZ algorithms across all settings, achieving F1-score gains of up to 27.2% over prior LLM-based approaches. The improvements are especially pronounced in challenging scenarios such as cross-file and ghost commits, with recall gains of up to 300% and 60%, respectively. Ablation studies show that task-specific tools and domain knowledge are critical, while compression tool outputs reduce token consumption by over 30% with negligible impact. The replication package is available.