Generative Modeling via Drifting

TL;DR

This paper introduces Drifting Models, a new generative modeling paradigm that evolves distributions during training, enabling high-quality one-step image generation with state-of-the-art results on ImageNet.

Contribution

The paper proposes Drifting Models, which incorporate a drifting field to evolve distributions during training, allowing for efficient one-step inference in generative modeling.

Findings

Achieved state-of-the-art FID scores of 1.54 in latent space and 1.61 in pixel space on ImageNet 256x256.

Demonstrated effective one-step generation with high image quality.

Introduced a new training objective based on a drifting field for distribution evolution.

Abstract

Generative modeling can be formulated as learning a mapping f such that its pushforward distribution matches the data distribution. The pushforward behavior can be carried out iteratively at inference time, for example in diffusion and flow-based models. In this paper, we propose a new paradigm called Drifting Models, which evolve the pushforward distribution during training and naturally admit one-step inference. We introduce a drifting field that governs the sample movement and achieves equilibrium when the distributions match. This leads to a training objective that allows the neural network optimizer to evolve the distribution. In experiments, our one-step generator achieves state-of-the-art results on ImageNet at 256 x 256 resolution, with an FID of 1.54 in latent space and 1.61 in pixel space. We hope that our work opens up new opportunities for high-quality one-step generation.