A Survey on Video Diffusion Models

TL;DR

This survey reviews the development and application of video diffusion models in AI-generated content, highlighting their evolution, key research areas, and future challenges in video generation, editing, and understanding.

Contribution

It provides the first comprehensive overview of video diffusion models, categorizing research areas and discussing future trends in this emerging field.

Findings

Diffusion models are increasingly replacing GANs and Transformers in video tasks.

Research is mainly focused on video generation, editing, and understanding.

Future challenges include scalability and real-world application integration.

Abstract

The recent wave of AI-generated content (AIGC) has witnessed substantial success in computer vision, with the diffusion model playing a crucial role in this achievement. Due to their impressive generative capabilities, diffusion models are gradually superseding methods based on GANs and auto-regressive Transformers, demonstrating exceptional performance not only in image generation and editing, but also in the realm of video-related research. However, existing surveys mainly focus on diffusion models in the context of image generation, with few up-to-date reviews on their application in the video domain. To address this gap, this paper presents a comprehensive review of video diffusion models in the AIGC era. Specifically, we begin with a concise introduction to the fundamentals and evolution of diffusion models. Subsequently, we present an overview of research on diffusion models in the video domain, categorizing the work into three key areas: video generation, video editing, and other video understanding tasks. We conduct a thorough review of the literature in these three key areas, including further categorization and practical contributions in the field. Finally, we discuss the challenges faced by research in this domain and outline potential future developmental trends. A comprehensive list of video diffusion models studied in this survey is available at https://github.com/ChenHsing/Awesome-Video-Diffusion-Models.