FMPlug: Plug-In Foundation Flow-Matching Priors for Inverse Problems

TL;DR

FMPlug is a new plug-in framework that improves foundation flow-matching priors for inverse problems by leveraging object similarity and Gaussianity, achieving superior results in image super-resolution and deblurring.

Contribution

It introduces a time-adaptive warm-up and Gaussianity regularization to enhance foundation FM priors in a domain-agnostic manner.

Findings

Outperforms state-of-the-art methods in image super-resolution.

Achieves significant improvements in Gaussian deblurring.

Effectively leverages foundation models for inverse problems.

Abstract

We present FMPlug, a novel plug-in framework that enhances foundation flow-matching (FM) priors for solving ill-posed inverse problems. Unlike traditional approaches that rely on domain-specific or untrained priors, FMPlug smartly leverages two simple but powerful insights: the similarity between observed and desired objects and the Gaussianity of generative flows. By introducing a time-adaptive warm-up strategy and sharp Gaussianity regularization, FMPlug unlocks the true potential of domain-agnostic foundation models. Our method beats state-of-the-art methods that use foundation FM priors by significant margins, on image super-resolution and Gaussian deblurring.