Dynamic Video Generation: Shaping Video Generation Across Time and Space

TL;DR

DVG is a dynamic framework for video generation that adaptively accelerates diffusion models across space and time, achieving significant speedups with minimal quality loss.

Contribution

It introduces a content-aware, automatic computation allocation method for efficient video diffusion, scalable across models and tasks without manual tuning.

Findings

Up to 7x speedup on HunyuanVideo models

18x acceleration with distillation

Near-lossless acceleration across diverse tasks

Abstract

Diffusion models have achieved impressive performance in video generation, but their iterative denoising process remains computationally expensive due to the large number of tokens processed at each timestep. Recently, progressive resolution sampling has emerged as a promising acceleration approach by reducing latent resolution in early stages. However, scaling this idea to video generation remains challenging, as the additional temporal dimension introduces diverse spatio-temporal demands across different videos, and compressing only a single dimension often leads to limited acceleration or degraded quality. Therefore, we propose DVG, a Dynamic Video Generation framework that jointly allocates computation across time and space, automatically selecting content-aware acceleration strategies without manual tuning or retraining. DVG achieves near-lossless acceleration across models and tasks, reaching up to 7 times speedup on HunyuanVideo and HunyuanVideo-1.5, and 18 times when combined with distillation, demonstrating its potential as a key component in today's large-scale efficient video generation systems. Our code is in supplementary material and will be released on Github.