A Methodology for Selecting and Composing Runtime Architecture Patterns for Production LLM Agents

TL;DR

This paper introduces a methodology for selecting and composing runtime architecture patterns for production LLM agents, emphasizing the stochastic-deterministic boundary as a core primitive and addressing reliability concerns.

Contribution

It presents a novel five-step methodology for pattern selection, a diagnostic procedure for failure analysis, and a new failure mode called replay divergence.

Findings

Identified six runtime patterns for LLM agents across different types.

Mapped production failures to specific pattern weaknesses.

Demonstrated the methodology on five workloads with a reference implementation.

Abstract

Production LLM agents combine stochastic model outputs with deterministic software systems, yet the boundary between the two is rarely treated as a first-class architectural object. This paper names that boundary the stochastic-deterministic boundary (SDB): a four-part contract among a proposer, verifier, commit step, and reject signal that specifies how an LLM output becomes a system action. We argue that the SDB is the load-bearing primitive of production agent runtimes. Around this primitive, we organize agent runtime design into three concerns: Coordination, State, and Control. We present a catalog of six runtime patterns that compose the SDB differently across conversational, autonomous, and long-horizon agents: hierarchical delegation, scatter-gather plus saga, event-driven sequencing, shared state machine, supervisor plus gate, and human in the loop. For each pattern, we trace its lineage to distributed-systems concepts and identify what changes when the worker is stochastic. The paper contributes a five-step methodology for selecting runtime patterns, a diagnostic procedure that maps production failures to pattern weaknesses, and a failure mode called replay divergence, in which LLM-based consumers of a deterministic event log produce different downstream outputs under model-version or prompt changes. A stylized reliability decomposition separates per-call model variance from architectural momentum, motivating the claim that as model variance decreases, pattern choice and SDB strength become increasingly important levers for long-run reliability. We apply the methodology to five workloads and provide one runnable reference implementation for a 90-day contract-renewal agent.