VIDM: Video Implicit Diffusion Models

TL;DR

This paper introduces VIDM, a novel video diffusion model that generates high-quality, diverse videos by modeling motion implicitly, outperforming previous GAN-based methods in quality and diversity.

Contribution

The paper proposes a new diffusion-based approach for video generation with implicit motion modeling and introduces strategies to enhance video quality.

Findings

Outperforms state-of-the-art GAN methods in FVD scores

Produces higher perceptual quality videos

Effective across various resolutions and frame counts

Abstract

Diffusion models have emerged as a powerful generative method for synthesizing high-quality and diverse set of images. In this paper, we propose a video generation method based on diffusion models, where the effects of motion are modeled in an implicit condition manner, i.e. one can sample plausible video motions according to the latent feature of frames. We improve the quality of the generated videos by proposing multiple strategies such as sampling space truncation, robustness penalty, and positional group normalization. Various experiments are conducted on datasets consisting of videos with different resolutions and different number of frames. Results show that the proposed method outperforms the state-of-the-art generative adversarial network-based methods by a significant margin in terms of FVD scores as well as perceptible visual quality.