OSV: One Step is Enough for High-Quality Image to Video Generation

TL;DR

This paper introduces OSV, a novel one-step video generation model that combines a two-stage training framework with a new discriminator design, achieving high-quality results with minimal inference steps.

Contribution

The work presents a unique two-stage training approach integrating consistency distillation and GAN training, along with a new video discriminator, enabling high-quality one-step video generation.

Findings

Outperforms existing methods on OpenWebVid-1M benchmark

Achieves 1-step FVD of 171.15, surpassing 8-step AnimateLCM

Approaches 25-step Stable Video Diffusion performance

Abstract

Video diffusion models have shown great potential in generating high-quality videos, making them an increasingly popular focus. However, their inherent iterative nature leads to substantial computational and time costs. While efforts have been made to accelerate video diffusion by reducing inference steps (through techniques like consistency distillation) and GAN training (these approaches often fall short in either performance or training stability). In this work, we introduce a two-stage training framework that effectively combines consistency distillation with GAN training to address these challenges. Additionally, we propose a novel video discriminator design, which eliminates the need for decoding the video latents and improves the final performance. Our model is capable of producing high-quality videos in merely one-step, with the flexibility to perform multi-step refinement for further performance enhancement. Our quantitative evaluation on the OpenWebVid-1M benchmark shows that our model significantly outperforms existing methods. Notably, our 1-step performance(FVD 171.15) exceeds the 8-step performance of the consistency distillation based method, AnimateLCM (FVD 184.79), and approaches the 25-step performance of advanced Stable Video Diffusion (FVD 156.94).