DDiT: Dynamic Resource Allocation for Diffusion Transformer Model Serving

TL;DR

DDiT is a system that optimizes resource allocation for diffusion transformer model serving, significantly improving latency and efficiency by dynamically balancing parallelism and resource use across model modules.

Contribution

The paper introduces DDiT, a novel resource management system with decoupled control and dynamic scheduling for diffusion transformer models, addressing inefficiencies in existing deployment methods.

Findings

Up to 1.44x reduction in p99 latency

Up to 1.43x reduction in average latency

Effective dynamic resource scaling across diverse datasets

Abstract

The Text-to-Video (T2V) model aims to generate dynamic and expressive videos from textual prompts. The generation pipeline typically involves multiple modules, such as language encoder, Diffusion Transformer (DiT), and Variational Autoencoders (VAE). Existing serving systems often rely on monolithic model deployment, while overlooking the distinct characteristics of each module, leading to inefficient GPU utilization. In addition, DiT exhibits varying performance gains across different resolutions and degrees of parallelism, and significant optimization potential remains unexplored. To address these problems, we present DDiT, a flexible system that integrates both inter-phase and intra-phase optimizations. DDiT focuses on two key metrics: optimal degree of parallelism, which prevents excessive parallelism for specific resolutions, and starvation time, which quantifies the sacrifice of each request. To this end, DDiT introduces a decoupled control mechanism to minimize the computational inefficiency caused by imbalances in the degree of parallelism between the DiT and VAE phases. It also designs a greedy resource allocation algorithm with a novel scheduling mechanism that operates at the single-step granularity, enabling dynamic and timely resource scaling. Our evaluation on the T5 encoder, OpenSora SDDiT, and OpenSora VAE models across diverse datasets reveals that DDiT significantly outperforms state-of-the-art baselines by up to 1.44x in p99 latency and 1.43x in average latency.