Treefix: Enabling Execution with a Tree of Prefixes

TL;DR

Treefix is a novel approach that uses large language models to iteratively generate and refine code prefixes, enabling more comprehensive execution of code snippets by creating a tree of prefixes that maximize executed lines.

Contribution

Treefix introduces a multi-step, feedback-driven method leveraging LLMs to iteratively build a tree of code prefixes, significantly improving execution coverage over existing approaches.

Findings

Achieves 25% and 7% more coverage than state-of-the-art in two datasets.

Covers 84% and 82% of code lines in the datasets.

Uses feedback to iteratively improve code execution coverage.

Abstract

The ability to execute code is a prerequisite for various dynamic program analyses. Learning-guided execution has been proposed as an approach to enable the execution of arbitrary code snippets by letting a neural model predict likely values for any missing variables. Although state-of-the-art learning-guided execution approaches, such as LExecutor, can enable the execution of a relative high amount of code, they are limited to predicting a restricted set of possible values and do not use any feedback from previous executions to execute even more code. This paper presents Treefix, a novel learning-guided execution approach that leverages LLMs to iteratively create code prefixes that enable the execution of a given code snippet. The approach addresses the problem in a multi-step fashion, where each step uses feedback about the code snippet and its execution to instruct an LLM to improve a previously generated prefix. This process iteratively creates a tree of prefixes, a subset of which is returned to the user as prefixes that maximize the number of executed lines in the code snippet. In our experiments with two datasets of Python code snippets, Treefix achieves 25% and 7% more coverage relative to the current state of the art in learning-guided execution, covering a total of 84% and 82% of all lines in the code snippets.