LLM as an Execution Estimator: Recovering Missing Dependency for Practical Time-travelling Debugging

TL;DR

This paper introduces RecovSlicing, a novel approach using large language models to accurately infer dynamic data dependencies in programs from partial traces, enabling efficient debugging without exhaustive instrumentation or multiple runs.

Contribution

RecovSlicing leverages LLMs to recover runtime variable definitions from partial traces, significantly improving dependency accuracy and recall in debugging tasks.

Findings

Achieves up to 98.3% accuracy in dependency recovery

Outperforms baseline methods in accuracy and recall

Identifies 16% more regressions in bug localization

Abstract

Determining the dynamic data dependency of a step that reads a variable $v$ is challenging. It typically requires either exhaustive instrumentation, which becomes prohibitively expensive when $v$ is defined within library calls, or repeated executions, which are impractical for non-deterministic programs. In this work, we propose RecovSlicing for computing dynamic data dependency in a single run, with only partial instrumentation. We explore the intuition that LLM can potentially infer program dynamics based on a partially recorded trace and relevant code as its context. Given (1) a partially recorded trace of a program $P$ and (2) the slicing criteria consisting of a query step $s$ and a query variable $v$ read by $s$, RecovSlicing computes the runtime definition of $v$ on the trace by estimating the miss-recorded execution of $P$. In this work, we allow the user to specify implicit query variable. Technically, built upon non-deterministic LLM, we address the challenges of (1) precise recovery of runtime variable value and structure from the recorded execution and (2) aligning the memory address of recovered variables and the recorded variables for definition analysis. We evaluate RecovSlicing on 8300 data dependencies across three slicing benchmarks, comparing it with Slicer4J, ND-Slicer, LLM Slicer, and re-execution Slicer. RecovSlicing achieves significantly higher accuracy (80.3%, 91.1%, 98.3%) and recall (up to 98.3%) than the best baseline (accuracy: 39.0%, 82.0%, 59.9%; recall: 53.4%, 79.1%, 87.1%). Integrated into a dual-slicing regression bug localizer, it identifies 16% more regressions.