Understanding, Accelerating, and Improving MeanFlow Training

TL;DR

This paper analyzes the training dynamics of MeanFlow, revealing how instantaneous and average velocity fields interact, and proposes an improved training scheme that accelerates convergence and enhances few-step generative quality.

Contribution

The paper provides a detailed analysis of MeanFlow's training dynamics and introduces a novel scheme that speeds up training and improves generation quality.

Findings

Faster convergence with the new training scheme.

Achieved state-of-the-art FID of 2.87 on ImageNet 256x256.

Reduced training time or model size while maintaining performance.

Abstract

MeanFlow promises high-quality generative modeling in few steps, by jointly learning instantaneous and average velocity fields. Yet, the underlying training dynamics remain unclear. We analyze the interaction between the two velocities and find: (i) well-established instantaneous velocity is a prerequisite for learning average velocity; (ii) learning of instantaneous velocity benefits from average velocity when the temporal gap is small, but degrades as the gap increases; and (iii) task-affinity analysis indicates that smooth learning of large-gap average velocities, essential for one-step generation, depends on the prior formation of accurate instantaneous and small-gap average velocities. Guided by these observations, we design an effective training scheme that accelerates the formation of instantaneous velocity, then shifts emphasis from short- to long-interval average velocity. Our enhanced MeanFlow training yields faster convergence and significantly better few-step generation: With the same DiT-XL backbone, our method reaches an impressive FID of 2.87 on 1-NFE ImageNet 256x256, compared to 3.43 for the conventional MeanFlow baseline. Alternatively, our method matches the performance of the MeanFlow baseline with 2.5x shorter training time, or with a smaller DiT-L backbone.