PIVOT: Bridging Planning and Execution in LLM Agents via Trajectory Refinement

TL;DR

PIVOT is a self-supervised framework that iteratively refines LLM-generated plans to improve execution success by addressing infeasibility and constraint violations, achieving state-of-the-art results.

Contribution

The paper introduces PIVOT, a novel trajectory refinement method that enhances plan-execution alignment in LLM agents through a four-stage, self-supervised process.

Findings

PIVOT achieves up to 94% improvement in constraint satisfaction with human feedback.

The autonomous version of PIVOT retains significant performance gains without external supervision.

PIVOT requires fewer tokens than competing methods, demonstrating computational efficiency.

Abstract

Large language model (LLM)-based agents frequently generate seemingly coherent plans that fail upon execution due to infeasible actions, constraint violations, and compounding errors over extended horizons. PIVOT (Plan-Inspect-eVOlve Trajectories) addresses this plan-execution misalignment through a self-supervised framework that treats trajectories as optimizable objects iteratively refined via environment interaction. The framework comprises four stages: PLAN generates candidate trajectories; INSPECT executes them and computes structured losses with textual gradients encoding plan-execution discrepancies; EVOLVE applies these signals to produce improved trajectories; and VERIFY performs a final global check against task constraints. A monotonic acceptance process ensures a non-decreasing solution quality. Empirical evaluations on DeepPlanning and GAIA demonstrate state-of-the-art performance: with human-in-the-loop (HITL) feedback, PIVOT establishes a strong upper bound up to 94% relative improvement in constraint satisfaction, while its fully autonomous variant retains substantial gains, showing that the core trajectory-refinement mechanism remains effective without external supervision. At the same time, PIVOT remains computationally efficient, requiring up to 3x to 5x fewer tokens than competing refinement methods. These findings establish that (self- or human-supervised) feedback-based trajectory optimization is a principled methodology for mitigating plan-execution gaps in autonomous agent systems.