Flow-Direct: Feedback-Efficient and Reusable Guidance for Flow Models via Non-Parametric Guidance Field

TL;DR

Flow-Direct introduces a persistent, non-parametric guidance field for flow models, significantly improving feedback efficiency and reusability in optimizing application-specific objectives using external reward functions.

Contribution

It provides a theoretically grounded, practical framework that leverages all collected samples to refine guidance, enabling reusable and multi-objective sample generation.

Findings

Guidance field derived from log-density ratio guides distribution transformation.

Empirical guidance field improves with more reward-evaluated samples.

Framework enables reusable guidance for generating samples satisfying multiple objectives.

Abstract

Training-free guidance enables pre-trained diffusion and flow models to optimize application-specific objectives using feedback from external black-box reward functions. However, existing methods are feedback-inefficient because reward feedback is used only transiently to inform a localized gradient approximation or a discrete search decision, and is subsequently discarded. To address this limitation, we propose Flow-Direct, a framework that guides the generation process via a persistent guidance field. Theoretically, this guidance field is analytically derived from the log-density ratio between the base and reward-weighted target distributions; it transports the pre-trained distribution to the target distribution. In practice, the field is implemented as a non-parametric estimator constructed from all accumulated reward-evaluated samples. As more samples are collected during optimization, this empirical guidance field becomes increasingly accurate. This persistent formulation yields two major advantages. First, Flow-Direct is highly feedback-efficient: because every evaluated sample is used to refine the global guidance field, no reward information is wasted. Second, the framework is naturally reusable: once optimization is complete, the collected dataset defines a reusable guidance field for generating novel target samples without additional reward evaluations, and distinct guidance fields can be combined to generate samples that simultaneously satisfy multiple objectives.