RunAgent: Interpreting Natural-Language Plans with Constraint-Guided Execution

TL;DR

RunAgent is a multi-agent platform that interprets natural-language plans, enforces constraints, and dynamically chooses reasoning methods to improve structured workflow execution accuracy.

Contribution

It introduces an agentic language with explicit control constructs and autonomous constraint validation to enhance LLM-based plan execution reliability.

Findings

RunAgent outperforms baseline LLMs on Natural-plan and SciBench datasets.

It effectively enforces stepwise constraints and validation during plan execution.

The system dynamically switches between reasoning, tool use, and code execution.

Abstract

Humans solve problems by executing targeted plans, yet large language models (LLMs) remain unreliable for structured workflow execution. We propose RunAgent, a multi-agent plan execution platform that interprets natural-language plans while enforcing stepwise execution through constraints and rubrics. RunAgent bridges the expressiveness of natural language with the determinism of programming via an agentic language with explicit control constructs (e.g., \texttt{IF}, \texttt{GOTO}, \texttt{FORALL}). Beyond verifying syntactic and semantic verification of the step output, which is performed based on the specific instruction of each step, RunAgent autonomously derives and validates constraints based on the description of the task and its instance at each step. RunAgent also dynamically selects among LLM-based reasoning, tool usage, and code generation and execution (e.g., in Python), and incorporates error correction mechanisms to ensure correctness. Finally, RunAgent filters the context history by retaining only relevant information during the execution of each step. Evaluations on Natural-plan and SciBench Datasets demonstrate that RunAgent outperforms baseline LLMs and state-of-the-art PlanGEN methods.