Adaptive Video Distillation: Mitigating Oversaturation and Temporal Collapse in Few-Step Generation

TL;DR

This paper introduces a specialized distillation framework for video diffusion models that reduces artifacts, mitigates temporal collapse, and improves efficiency in few-step video generation, outperforming existing methods.

Contribution

The paper presents a novel video-specific distillation approach with adaptive loss functions and inference strategies, addressing oversaturation and temporal collapse issues in video diffusion models.

Findings

Achieves stable few-step video synthesis with high perceptual quality.

Significantly outperforms existing distillation baselines on benchmarks.

Reduces sampling overhead while maintaining motion realism.

Abstract

Video generation has recently emerged as a central task in the field of generative AI. However, the substantial computational cost inherent in video synthesis makes model distillation a critical technique for efficient deployment. Despite its significance, there is a scarcity of methods specifically designed for video diffusion models. Prevailing approaches often directly adapt image distillation techniques, which frequently lead to artifacts such as oversaturation, temporal inconsistency, and mode collapse. To address these challenges, we propose a novel distillation framework tailored specifically for video diffusion models. Its core innovations include: (1) an adaptive regression loss that dynamically adjusts spatial supervision weights to prevent artifacts arising from excessive distribution shifts; (2) a temporal regularization loss to counteract temporal collapse, promoting smooth and physically plausible sampling trajectories; and (3) an inference-time frame interpolation strategy that reduces sampling overhead while preserving perceptual quality. Extensive experiments and ablation studies on the VBench and VBench2 benchmarks demonstrate that our method achieves stable few-step video synthesis, significantly enhancing perceptual fidelity and motion realism. It consistently outperforms existing distillation baselines across multiple metrics.