Diffusion Adversarial Post-Training for One-Step Video Generation

TL;DR

This paper introduces Seaweed-APT, a novel adversarial post-training method that enables real-time, one-step high-resolution video and image generation with quality comparable to state-of-the-art models.

Contribution

It presents a new adversarial post-training approach for diffusion models that significantly improves generation speed and quality for videos and images in a single step.

Findings

Generates 2-second, 1280x720 videos at 24fps in real time.

Produces 1024px images with quality comparable to top methods.

Enhances training stability and quality through architectural and procedural improvements.

Abstract

The diffusion models are widely used for image and video generation, but their iterative generation process is slow and expansive. While existing distillation approaches have demonstrated the potential for one-step generation in the image domain, they still suffer from significant quality degradation. In this work, we propose Adversarial Post-Training (APT) against real data following diffusion pre-training for one-step video generation. To improve the training stability and quality, we introduce several improvements to the model architecture and training procedures, along with an approximated R1 regularization objective. Empirically, our experiments show that our adversarial post-trained model, Seaweed-APT, can generate 2-second, 1280x720, 24fps videos in real time using a single forward evaluation step. Additionally, our model is capable of generating 1024px images in a single step, achieving quality comparable to state-of-the-art methods.