AAFLOW: Scalable Patterns for Agentic AI Workflows

TL;DR

AAFLOW introduces a scalable, efficient distributed runtime for agentic AI workflows, significantly improving data flow and execution speed in large language model systems.

Contribution

It presents a formal execution model using operator abstraction, leveraging Apache Arrow and Cylon for high-performance, communication-efficient workflows.

Findings

Up to 4.64 times pipeline speedup

2.8 times gains in embedding and upsert phases

Retains comparable LLM generation throughput

Abstract

Agentic workflows in large language model systems integrate retrieval, reasoning, and memory, but existing frameworks suffer from scalability and reproducibility limitations due to fragmented data orchestration, serialization overhead, and non-deterministic execution. Although these frameworks increase flexibility, they don't have a formal execution model that adheres to the principles of high-performance computing. We introduce AAFLOW, a unified distributed runtime that creates communication-efficient execution plans by modeling agentic workflows as an operator abstraction. Using Apache Arrow and Cylon, AAFLOW creates a zero-copy data plane that allows direct interoperability between preprocessing, embedding, and vector retrieval without the need for serialization overhead. To lower coordination costs, it uses resource-deterministic scheduling and asynchronous batching. While retaining comparable LLM generation throughput, experimental results demonstrate up to 4.64 times pipeline speedup and 2.8 times gains in embedding and upsert phases. Rather than LLM inference acceleration, these advantages result from enhanced data flow, batching, and communication efficiency.