EnCompass: Enhancing Agent Programming with Search Over Program Execution Paths

TL;DR

EnCompass introduces a novel framework for agent programming that separates core workflow logic from inference strategies, enabling flexible experimentation and improved reliability of LLM-based agents.

Contribution

The paper presents PAN, a programming model that disentangles workflow and inference strategies, implemented in Python as the EnCompass framework for flexible agent development.

Findings

EnCompass allows quick switching between inference strategies.

Framework improves agent reliability with minimal code changes.

Demonstrated through three case studies.

Abstract

We introduce a new approach to agent programming, the development of LLM-based agents. Current approaches to agent programming often entangle two aspects of agent design: the core workflow logic and the inference-time strategy (e.g., tree search). We introduce "probabilistic angelic nondeterminism" ("PAN"), a programming model that disentangles these two concerns, allowing the programmer to describe the agent workflow and independently experiment with different inference-time strategies by simply changing a few inputs. We provide an implementation of PAN in Python as the EnCompass framework, which uses a Python decorator to compile agent workflow programs into a search space. We present three case studies that demonstrate how the framework lets the programmer quickly improve the reliability of an agent and easily switch between different inference-time strategies, all with little additional coding.