vLLM-Omni: Fully Disaggregated Serving for Any-to-Any Multimodal Models

TL;DR

vLLM-Omni introduces a fully disaggregated serving system that efficiently handles complex any-to-any multimodal models by decomposing architectures into interconnected stages, significantly improving performance and resource utilization.

Contribution

It proposes a novel stage abstraction and disaggregated backend enabling flexible, efficient serving of complex multimodal models with multiple interconnected components.

Findings

Reduces job completion time by up to 91.4%.

Supports flexible GPU allocation and per-stage batching.

Enables decomposition of complex architectures into interconnected stages.

Abstract

Any-to-any multimodal models that jointly handle text, images, video, and audio represent a significant advance in multimodal AI. However, their complex architectures (typically combining multiple autoregressive LLMs, diffusion transformers, and other specialized components) pose substantial challenges for efficient model serving. Existing serving systems are mainly tailored to a single paradigm, such as autoregressive LLMs for text generation or diffusion transformers for visual generation. They lack support for any-to-any pipelines that involve multiple interconnected model components. As a result, developers must manually handle cross-stage interactions, leading to huge performance degradation. We present vLLM-Omni, a fully disaggregated serving system for any-to-any models. vLLM-Omni features a novel stage abstraction that enables users to decompose complex any-to-any architectures into interconnected stages represented as a graph, and a disaggregated stage execution backend that optimizes resource utilization and throughput across stages. Each stage is independently served by an LLM or diffusion engine with per-stage request batching, flexible GPU allocation, and unified inter-stage connectors for data routing. Experimental results demonstrate that vLLM-Omni reduces job completion time (JCT) by up to 91.4% compared to baseline methods. The code is public available at https://github.com/vllm-project/vllm-omni.