Greed is Good: A Unifying Perspective on Guided Generation

TL;DR

This paper unifies posterior and end-to-end guidance techniques in flow/diffusion models, providing theoretical insights and a method to interpolate between them for improved control in generative tasks.

Contribution

It reveals the theoretical connection between posterior and end-to-end guidance, and introduces an interpolation method to balance compute and accuracy.

Findings

Unified guidance techniques through theoretical analysis

Interpolation method improves guidance trade-offs

Validated on inverse image and molecular generation tasks

Abstract

Training-free guided generation is a widely used and powerful technique that allows the end user to exert further control over the generative process of flow/diffusion models. Generally speaking, two families of techniques have emerged for solving this problem for gradient-based guidance: namely, posterior guidance (i.e., guidance via projecting the current sample to the target distribution via the target prediction model) and end-to-end guidance (i.e., guidance by performing backpropagation throughout the entire ODE solve). In this work, we show that these two seemingly separate families can actually be unified by looking at posterior guidance as a greedy strategy of end-to-end guidance. We explore the theoretical connections between these two families and provide an in-depth theoretical of these two techniques relative to the continuous ideal gradients. Motivated by this analysis we then show a method for interpolating between these two families enabling a trade-off between compute and accuracy of the guidance gradients. We then validate this work on several inverse image problems and property-guided molecular generation.