CHAI: CacHe Attention Inference for text2video

TL;DR

CHAI introduces Cache Attention to accelerate text-to-video diffusion models by reusing shared scene information, achieving 1.65x to 3.35x faster inference without quality loss.

Contribution

The paper proposes Cache Attention, a novel cross-inference caching method that significantly speeds up text-to-video diffusion without retraining or quality degradation.

Findings

Achieves high-quality video generation with as few as 8 denoising steps.

Speeds up inference by 1.65x to 3.35x compared to baseline.

Maintains video quality while reducing latency.

Abstract

Text-to-video diffusion models deliver impressive results but remain slow because of the sequential denoising of 3D latents. Existing approaches to speed up inference either require expensive model retraining or use heuristic-based step skipping, which struggles to maintain video quality as the number of denoising steps decreases. Our work, CHAI, aims to use cross-inference caching to reduce latency while maintaining video quality. We introduce Cache Attention as an effective method for attending to shared objects/scenes across cross-inference latents. This selective attention mechanism enables effective reuse of cached latents across semantically related prompts, yielding high cache hit rates. We show that it is possible to generate high-quality videos using Cache Attention with as few as 8 denoising steps. When integrated into the overall system, CHAI is 1.65x - 3.35x faster than baseline OpenSora 1.2 while maintaining video quality.