FlowAgent: Achieving Compliance and Flexibility for Workflow Agents

TL;DR

FlowAgent introduces a framework that combines natural language and code to enable LLM-based workflow agents to maintain procedural compliance while flexibly handling unexpected queries, improving real-world automation.

Contribution

We propose FlowAgent, a novel framework with Procedure Description Language (PDL) that balances compliance and flexibility in LLM workflow agents, and a new evaluation methodology for OOW query handling.

Findings

FlowAgent effectively manages out-of-workflow queries.

It maintains high compliance with predefined procedures.

Experiments show improved flexibility and adherence across datasets.

Abstract

The integration of workflows with large language models (LLMs) enables LLM-based agents to execute predefined procedures, enhancing automation in real-world applications. Traditional rule-based methods tend to limit the inherent flexibility of LLMs, as their predefined execution paths restrict the models' action space, particularly when the unexpected, out-of-workflow (OOW) queries are encountered. Conversely, prompt-based methods allow LLMs to fully control the flow, which can lead to diminished enforcement of procedural compliance. To address these challenges, we introduce FlowAgent, a novel agent framework designed to maintain both compliance and flexibility. We propose the Procedure Description Language (PDL), which combines the adaptability of natural language with the precision of code to formulate workflows. Building on PDL, we develop a comprehensive framework that empowers LLMs to manage OOW queries effectively, while keeping the execution path under the supervision of a set of controllers. Additionally, we present a new evaluation methodology to rigorously assess an LLM agent's ability to handle OOW scenarios, going beyond routine flow compliance tested in existing benchmarks. Experiments on three datasets demonstrate that FlowAgent not only adheres to workflows but also effectively manages OOW queries, highlighting its dual strengths in compliance and flexibility. The code is available at https://github.com/Lightblues/FlowAgent.