Software-Defined Agentic Serving

TL;DR

This paper introduces a programmable, system-aware agentic serving framework inspired by SDN, designed to improve control, responsiveness, and efficiency in multi-agent LLM pipelines under dynamic conditions.

Contribution

It proposes a novel SDN-inspired architecture for agentic serving, enabling runtime control and high-level intent-driven management of multi-agent systems.

Findings

Enables dynamic control of communication attributes based on runtime state.

Improves responsiveness and efficiency in multi-agent LLM serving.

Facilitates high-level intent-driven agentic serving.

Abstract

As multi-agent LLM pipelines grow in complexity, existing serving paradigms fail to adapt to the dynamic serving conditions. We argue that agentic serving systems should be programmable and system-aware, unlike existing serving which statically encode the parameters. In this work, we propose a new SDN-inspired agentic serving framework that helps control the key attributes of communication based on runtime state. This architecture enables serving-efficient, responsive agent systems and paves the way for high-level intent-driven agentic serving.