Adaptive Hybrid Caching for Efficient Text-to-Video Diffusion Model Acceleration

TL;DR

This paper introduces MixCache, a flexible, training-free caching framework that adaptively optimizes caching granularity to accelerate video diffusion models without compromising quality.

Contribution

It presents a novel adaptive hybrid caching strategy that dynamically balances quality and speed in video diffusion model inference, surpassing existing single-granularity methods.

Findings

Achieves up to 1.97x speedup in video generation

Maintains high generation quality with improved efficiency

Demonstrates effectiveness across diverse models

Abstract

Efficient video generation models are increasingly vital for multimedia synthetic content generation. Leveraging the Transformer architecture and the diffusion process, video DiT models have emerged as a dominant approach for high-quality video generation. However, their multi-step iterative denoising process incurs high computational cost and inference latency. Caching, a widely adopted optimization method in DiT models, leverages the redundancy in the diffusion process to skip computations in different granularities (e.g., step, cfg, block). Nevertheless, existing caching methods are limited to single-granularity strategies, struggling to balance generation quality and inference speed in a flexible manner. In this work, we propose MixCache, a training-free caching-based framework for efficient video DiT inference. It first distinguishes the interference and boundary between different caching strategies, and then introduces a context-aware cache triggering strategy to determine when caching should be enabled, along with an adaptive hybrid cache decision strategy for dynamically selecting the optimal caching granularity. Extensive experiments on diverse models demonstrate that, MixCache can significantly accelerate video generation (e.g., 1.94$\times$ speedup on Wan 14B, 1.97$\times$ speedup on HunyuanVideo) while delivering both superior generation quality and inference efficiency compared to baseline methods.