On the Guidance of Flow Matching

TL;DR

This paper introduces a comprehensive framework for guiding flow matching in generative tasks, offering new guidance techniques and theoretical insights, with demonstrated effectiveness across various applications.

Contribution

It presents the first general guidance framework for flow matching, including novel training-free, training-based, and approximate guidance methods, along with theoretical analysis.

Findings

Effective guidance methods demonstrated on synthetic and real datasets

Theoretical guidelines for selecting guidance techniques in different scenarios

Verification of the framework's correctness through experiments

Abstract

Flow matching has shown state-of-the-art performance in various generative tasks, ranging from image generation to decision-making, where generation under energy guidance (abbreviated as guidance in the following) is pivotal. However, the guidance of flow matching is more general than and thus substantially different from that of its predecessor, diffusion models. Therefore, the challenge in guidance for general flow matching remains largely underexplored. In this paper, we propose the first framework of general guidance for flow matching. From this framework, we derive a family of guidance techniques that can be applied to general flow matching. These include a new training-free asymptotically exact guidance, novel training losses for training-based guidance, and two classes of approximate guidance that cover classical gradient guidance methods as special cases. We theoretically investigate these different methods to give a practical guideline for choosing suitable methods in different scenarios. Experiments on synthetic datasets, image inverse problems, and offline reinforcement learning demonstrate the effectiveness of our proposed guidance methods and verify the correctness of our flow matching guidance framework. Code to reproduce the experiments can be found at https://github.com/AI4Science-WestlakeU/flow_guidance.