GraphFlow: A Graph-Based Workflow Management for Efficient LLM-Agent Serving

TL;DR

GraphFlow introduces a graph-based workflow management system for LLM agents that enhances efficiency and generalization by dynamically constructing workflows and optimizing memory usage.

Contribution

The paper presents wGraph, a unified graph representation for workflows, and introduces GraphFlow, a system that improves LLM-agent serving through adaptive workflow generation and efficient state management.

Findings

Outperforms state-of-the-art methods on five benchmarks by ~4.95 percentage points.

Achieves ~4× reduction in memory footprint during agent serving.

Demonstrates improved generalization to unseen tasks.

Abstract

Large Language Model (LLM)-based agents demonstrate strong reasoning and execution capabilities on complex tasks when guided by structured instructions, commonly referred to as workflows. However, existing workflow-assisted agent serving systems typically rely on predefined templates and shallow matching mechanisms, which limit their ability to capture deep semantic relationships and generalize to previously unseen tasks. To address these limitations, we propose a new workflow management paradigm that represents workflows using a unified graph, termed wGraph, where each node corresponds to an atomic operation. wGraph serves as a shared substrate from which task-specific workflows are dynamically instantiated. Building on wGraph primitives, we introduce GraphFlow, a system that efficiently integrates workflows into agent serving through two key designs. First, adaptive workflow generation dynamically constructs workflows from wGraph based on task semantics and constraint requirements. Second, workflow state management exploits wGraph structure to efficiently manage Key-Value (KV) caches, reducing redundant computation during agent serving. Extensive experiments across five benchmark datasets show that GraphFlow consistently outperforms state-of-the-art methods, yielding an average performance improvement of approximately 4.95 percentage points, while achieving an approximately 4$\times$ reduction in memory footprint.