SemLoc: Structured Grounding of Free-Form LLM Reasoning for Fault Localization

TL;DR

SemLoc is a novel fault localization framework that leverages structured semantic grounding of LLM reasoning to accurately identify program faults, especially semantic bugs, by converting reasoning into verifiable constraints.

Contribution

It introduces a structured semantic grounding approach that converts LLM reasoning into a verifiable intermediate representation for fault localization.

Findings

SemLoc achieves 42.8% Top-1 accuracy on SemFault-250.

It reduces inspection to 7.6% of executable lines.

Counterfactual verification adds 12% accuracy gain.

Abstract

Fault localization identifies program locations responsible for observed failures. Existing techniques rank suspicious code using syntactic spectra--signals derived from execution structure such as statement coverage, control-flow divergence, or dependency reachability. These signals collapse for semantic bugs, where failing and passing executions follow identical code paths and differ only in whether semantic intent is satisfied. Recent LLM-based approaches introduce semantic reasoning but produce stochastic, unverifiable outputs that cannot be systematically cross-referenced across tests or distinguish root causes from cascading effects. We present SemLoc, a fault localization framework based on structured semantic grounding. SemLoc converts free-form LLM reasoning into a closed intermediate representation that binds each inferred property to a typed program anchor, enabling runtime checking and attribution to program structure. It executes instrumented programs to construct a semantic violation spectrum--a constraint-by-test matrix--from which suspiciousness scores are derived analogously to coverage-based methods. A counterfactual verification step further prunes over-approximate constraints and isolates primary causal violations. We evaluate SemLoc on SemFault-250, a corpus of 250 Python programs with single semantic faults. SemLoc outperforms five coverage-, reduction-, and LLM-based baselines, achieving Top-1 accuracy of 42.8% and Top-3 of 68%, while reducing inspection to 7.6% of executable lines. Counterfactual verification provides an additional 12% accuracy gain and identifies primary causal semantic constraints.