Frontier: Simulating the Next Generation of LLM Inference Systems

TL;DR

Frontier is a high-fidelity simulator tailored for next-generation LLM inference systems, supporting complex architectures like MoE and disaggregated models, enabling better design and optimization of scalable inference workflows.

Contribution

It introduces a unified, high-accuracy simulation framework for both co-located and disaggregated LLM inference architectures, including MoE with expert parallelism.

Findings

Supports simulation of complex routing and pipelining strategies

Provides refined operator models for higher fidelity

Enables community-driven optimization of LLM inference systems

Abstract

Large Language Model (LLM) inference is growing increasingly complex with the rise of Mixture-of-Experts (MoE) models and disaggregated architectures that decouple components like prefill/decode (PD) or attention/FFN (AF) for heterogeneous scaling. Existing simulators, architected for co-located, dense models, are unable to capture the intricate system dynamics of these emerging paradigms. We present Frontier, a high-fidelity simulator designed from the ground up for this new landscape. Frontier introduces a unified framework to model both co-located and disaggregated systems, providing native support for MoE inference with expert parallelism (EP). It enables the simulation of complex workflows like cross-cluster expert routing and advanced pipelining strategies for latency hiding. To ensure fidelity and usability, Frontier incorporates refined operator models for improved accuracy. Frontier empowers the community to design and optimize the future of LLM inference at scale.