Improving Classifier-Free Guidance of Flow Matching via Manifold Projection

TL;DR

This paper offers a principled, optimization-based reinterpretation of classifier-free guidance in flow matching models, introducing a manifold projection method with Anderson Acceleration to enhance controllable generation.

Contribution

It reformulates CFG as a homotopy optimization with manifold constraints and proposes a training-free, iterative projection method with acceleration for improved stability and fidelity.

Findings

Enhanced generation fidelity and prompt alignment.

Improved robustness to guidance scale variations.

Validated on large-scale models like DiT-XL-2-256 and Stable Diffusion 3.5.

Abstract

Classifier-free guidance (CFG) is a widely used technique for controllable generation in diffusion and flow-based models. Despite its empirical success, CFG relies on a heuristic linear extrapolation that is often sensitive to the guidance scale. In this work, we provide a principled interpretation of CFG through the lens of optimization. We demonstrate that the velocity field in flow matching corresponds to the gradient of a sequence of smoothed distance functions, which guides latent variables toward the scaled target image set. This perspective reveals that the standard CFG formulation is an approximation of this gradient, where the prediction gap, the discrepancy between conditional and unconditional outputs, governs guidance sensitivity. Leveraging this insight, we reformulate the CFG sampling as a homotopy optimization with a manifold constraint. This formulation necessitates a manifold projection step, which we implement via an incremental gradient descent scheme during sampling. To improve computational efficiency and stability, we further enhance this iterative process with Anderson Acceleration without requiring additional model evaluations. Our proposed methods are training-free and consistently refine generation fidelity, prompt alignment, and robustness to the guidance scale. We validate their effectiveness across diverse benchmarks, demonstrating significant improvements on large-scale models such as DiT-XL-2-256, Flux, and Stable Diffusion 3.5.