Analytical Provisioning for Attention-FFN Disaggregated LLM Serving under Stochastic Workloads

TL;DR

This paper develops an analytical framework for optimally provisioning Attention-FFN disaggregated LLM serving systems under stochastic workloads, balancing memory and compute resources to minimize idle time and blocking.

Contribution

It introduces a novel workload-driven provisioning model with a closed-form optimal ratio and a Gaussian refinement, validated by trace-calibrated simulations.

Findings

Predicted optimal A/F ratio matches simulation within 10%.

Workload statistic θ governs provisioning across distributions.

Framework decomposes bottlenecks into Attention, communication, and FFN regimes.

Abstract

Attentio-FFN disaggregation (AFD) is an emerging architecture for LLM decoding that separates state-heavy, KV-cache-dominated Attention computation from stateless, compute-intensive FFN computation, connected by per-step communication. While AFD enables independent scaling of memory and compute resources, its performance is highly sensitive to the Attention/FFN provisioning ratio: mis-sizing induces step-level blocking and costly device idle time. We develop an analytical provisioning framework for AFD bundles in an $r$A--$1$F topology under stochastic workloads. Two sources of randomness shape the problem: per-slot Attention workload evolves as KV caches grow and completed requests are replenished with random prompt and decode lengths, and synchronized execution across Attention workers introduces a barrier governed by the slowest worker. We address both via a renewal-reward characterization of the per-slot stationary token load, identifying a single workload statistic $\theta$ that governs provisioning under arbitrary prefill-decode distributions and admits a nonparametric estimator from request traces. The analysis yields a closed-form mean-field rule for the optimal A/F ratio decomposing into Attention-, communication-, and FFN-bottleneck regimes, together with a Gaussian barrier-aware refinement that quantifies cross-worker synchronization overhead. A trace-calibrated AFD simulator supports the framework across workloads: the predicted optimal ratio matches the simulation-optimal within 10%. Together, these results provide a compact, calibratable account of how stochastic workload structure determines provisioning in disaggregated LLM serving.