Trajectory Stitching for Solving Inverse Problems with Flow-Based Models

TL;DR

MS-Flow introduces a trajectory-based approach with intermediate latent states for flow models, reducing memory costs and enhancing inverse problem solutions like inpainting and super-resolution.

Contribution

It proposes MS-Flow, a novel method that models the generative trajectory with intermediate states, improving efficiency and reconstruction quality in inverse problems.

Findings

MS-Flow reduces memory usage compared to traditional methods.

MS-Flow achieves better reconstruction quality in image recovery tasks.

Demonstrated effectiveness on inpainting, super-resolution, and CT reconstruction.

Abstract

Flow-based generative models have emerged as powerful priors for solving inverse problems. One option is to directly optimize the initial latent code (noise), such that the flow output solves the inverse problem. However, this requires backpropagating through the entire generative trajectory, incurring high memory costs and numerical instability. We propose MS-Flow, which represents the trajectory as a sequence of intermediate latent states rather than a single initial code. By enforcing the flow dynamics locally and coupling segments through trajectory-matching penalties, MS-Flow alternates between updating intermediate latent states and enforcing consistency with observed data. This reduces memory consumption while improving reconstruction quality. We demonstrate the effectiveness of MS-Flow over existing methods on image recovery and inverse problems, including inpainting, super-resolution, and computed tomography.