StackPilot: Autonomous Function Agents for Scalable and Environment-Free Code Execution

TL;DR

StackPilot is a novel multi-agent framework that enables language-agnostic, environment-free code verification and execution, significantly improving reliability over traditional environment-dependent methods.

Contribution

It introduces a Function-as-Agents paradigm, an LLM-as-Executor strategy, and a snapshot mechanism for deterministic context switching, advancing automated code verification.

Findings

Achieves 89%-97% reliability in code verification.

Outperforms baseline approaches in diverse programming tasks.

Operates independently of conventional toolchains.

Abstract

Recent advances in large language models (LLMs) have substantially enhanced automated code generation across a wide range of programming languages. Nonetheless, verifying the correctness and executability of LLM-generated code remains a significant challenge, as traditional methods rely on language-specific compilers and environment-dependent runtimes. To overcome these limitations, we introduce StackPilot, an LLM-native, multi-agent framework designed for language-agnostic code verification and execution, which operates independently of conventional toolchains. StackPilot offers three principal innovations: (1) a Function-as-Agents paradigm, in which each function is modeled as an autonomous agent capable of fine-grained reasoning and collaborative verification; (2) an LLM-as-Executor strategy, which enables scalable verification via stack-based scheduling; and (3) a novel snapshot mechanism that preserves complete execution contexts, facilitating deterministic and lossless context switching during verification. Empirical evaluations demonstrate that StackPilot achieves framework reliability rates between 89% and 97%, substantially outperforming baseline approaches. These results indicate that StackPilot can reliably verify and execute a significantly larger proportion of LLM-generated code across diverse programming tasks compared to existing methods.