Discrete Meanflow Training Curriculum

TL;DR

This paper introduces a Discrete Meanflow (DMF) training curriculum that significantly reduces training time and computational resources for Meanflow models, enabling efficient one-step image generation.

Contribution

The paper proposes a novel discretization of the Meanflow objective that improves training efficiency and stability, especially when fine-tuning from pretrained Flow Models.

Findings

DMF curriculum achieves 1-step FID 3.36 on CIFAR-10 in 2000 epochs.

Discretization yields a consistency property that enhances training stability.

Fine-tuning from pretrained models accelerates Meanflow training.

Abstract

Flow-based image generative models exhibit stable training and produce high quality samples when using multi-step sampling procedures. One-step generative models can produce high quality image samples but can be difficult to optimize as they often exhibit unstable training dynamics. Meanflow models exhibit excellent few-step sampling performance and tantalizing one-step sampling performance. Notably, MeanFlow models that achieve this have required extremely large training budgets. We significantly decrease the amount of computation and data budget it takes to train Meanflow models by noting and exploiting a particular discretization of the Meanflow objective that yields a consistency property which we formulate into a ``Discrete Meanflow'' (DMF) Training Curriculum. Initialized with a pretrained Flow Model, DMF curriculum reaches one-step FID 3.36 on CIFAR-10 in only 2000 epochs. We anticipate that faster training curriculums of Meanflow models, specifically those fine-tuned from existing Flow Models, drives efficient training methods of future one-step examples.