DiffServe: Efficiently Serving Text-to-Image Diffusion Models with Query-Aware Model Scaling

TL;DR

DiffServe introduces a query-aware model scaling system for text-to-image diffusion models, enabling efficient serving by dynamically adjusting model complexity based on query difficulty, resulting in improved response quality and reduced latency.

Contribution

The paper presents a novel end-to-end system that constructs model cascades and allocates resources dynamically for efficient diffusion model serving.

Findings

Up to 24% improvement in response quality

19-70% lower latency violation rates

Effective handling of variable query demands

Abstract

Text-to-image generation using diffusion models has gained increasing popularity due to their ability to produce high-quality, realistic images based on text prompts. However, efficiently serving these models is challenging due to their computation-intensive nature and the variation in query demands. In this paper, we aim to address both problems simultaneously through query-aware model scaling. The core idea is to construct model cascades so that easy queries can be processed by more lightweight diffusion models without compromising image generation quality. Based on this concept, we develop an end-to-end text-to-image diffusion model serving system, DiffServe, which automatically constructs model cascades from available diffusion model variants and allocates resources dynamically in response to demand fluctuations. Our empirical evaluations demonstrate that DiffServe achieves up to 24% improvement in response quality while maintaining 19-70% lower latency violation rates compared to state-of-the-art model serving systems.