Saddle-Free Guidance: Improved On-Manifold Sampling without Labels or Additional Training

TL;DR

This paper introduces saddle-free guidance (SFG), a novel method for on-manifold sampling in score-based generative models that does not require labels or additional training, improving diversity and quality of generated images.

Contribution

SFG leverages the positive curvature in saddle regions of log density estimates to guide models, achieving state-of-the-art results without extra training or labeled data.

Findings

SFG achieves state-of-the-art FID and FD-DINOv2 metrics in unconditional ImageNet-512 generation.

Combining SFG with Auto-Guidance yields top FD-DINOv2 scores in unconditional sampling.

SFG enhances diversity of generated images while maintaining fidelity and prompt adherence.

Abstract

Score-based generative models require guidance in order to generate plausible, on-manifold samples. The most popular guidance method, Classifier-Free Guidance (CFG), is only applicable in settings with labeled data and requires training an additional unconditional score-based model. More recently, Auto-Guidance adopts a smaller, less capable version of the original model to guide generation. While each method effectively promotes the fidelity of generated data, each requires labeled data or the training of additional models, making it challenging to guide score-based models when (labeled) training data are not available or training new models is not feasible. We make the surprising discovery that the positive curvature of log density estimates in saddle regions provides strong guidance for score-based models. Motivated by this, we develop saddle-free guidance (SFG) which maintains estimates of maximal positive curvature of the log density to guide individual score-based models. SFG has the same computational cost of classifier-free guidance, does not require additional training, and works with off-the-shelf diffusion and flow matching models. Our experiments indicate that SFG achieves state-of-the-art FID and FD-DINOv2 metrics in single-model unconditional ImageNet-512 generation. When SFG is combined with Auto-Guidance, its unconditional samples achieve general state-of-the-art in FD-DINOv2 score. Our experiments with FLUX.1-dev and Stable Diffusion v3.5 indicate that SFG boosts the diversity of output images compared to CFG while maintaining excellent prompt adherence and image fidelity.