Saving Foundation Flow-Matching Priors for Inverse Problems

TL;DR

FMPlug is a new framework that enhances foundation flow-matching models for inverse problems by adding problem-specific guidance, making them more practical and effective especially in scientific applications with limited data.

Contribution

The paper introduces FMPlug, a plug-in method that improves foundation flow-matching models for inverse problems by combining warm-start strategies and Gaussianity regularization.

Findings

FMPlug outperforms existing methods in image restoration tasks.

It enables foundation flow-matching models to be more effective with few samples.

Experimental results demonstrate superior performance in scientific inverse problems.

Abstract

Foundation flow-matching (FM) models promise universal priors for solving inverse problems (IPs); yet today, they trail behind domain-specific and even untrained priors. \emph{How can we unlock their potential?} We introduce FMPlug, a plug-in framework that redefines how foundation FMs are used in IPs. FMPlug combines an instance-guided, time-dependent warm-start strategy with sharp Gaussianity regularization, adding problem-specific guidance while preserving the Gaussian structures. For evaluation, we consider both simple image restoration tasks and scientific IPs with a few similar samples -- where the prohibitive cost of data collection and model training hinders the development of domain-specific generative models. Our superior experimental results confirm the effectiveness of FMPlug. Overall, FMPlug paves the way for making foundation FM models practical, reusable priors for IPs, especially scientific ones with few similar samples. More details are available at https://sun-umn.github.io/xm-plug/ .