A Flow Matching Algorithm for Many-Shot Adaptation to Unseen Distributions

TL;DR

The paper introduces FP-FM, a flow matching algorithm that enables efficient adaptation to unseen distributions by learning basis functions and projecting onto them, improving sample quality without retraining.

Contribution

It proposes a novel method for model adaptation to new distributions using basis functions and least-squares projection, enhancing flexibility and efficiency.

Findings

FP-FM achieves higher precision and recall than baselines.

It performs well on synthetic and image datasets.

Strong gains on unseen distributions.

Abstract

While generative modeling has achieved remarkable success on tasks like natural language-conditioned image generation, enabling model adaptation from example data points remains a relatively underexplored and challenging problem. To this end, we propose Function Projection for Flow Matching (FP-FM), an algorithm that directly conditions generation on samples from the target distribution. FP-FM learns basis functions to span the velocity fields corresponding to a set of training distributions, and adapts to new distributions by computing a simple least-squares projection onto this basis. This enables efficient generation of samples from diverse target distributions without additional training at inference time. We further introduce multiple variants of FP-FM that provide a trade-off in expressivity and compute by enriching the coefficient calculation, e.g., by making the coefficients dependent on time. FP-FM achieves greatly improved precision and recall relative to baselines across synthetic and image-based datasets, with especially strong gains on unseen distributions.