Improving Rectified Flow with Boundary Conditions

TL;DR

This paper introduces a Boundary RF Model that enforces boundary conditions in Rectified Flow, significantly improving generative modeling accuracy and FID scores on ImageNet.

Contribution

The paper proposes a boundary-enforced approach for Rectified Flow that ensures velocity fields satisfy boundary conditions with minimal code changes.

Findings

8.01% FID improvement on ImageNet with ODE sampling

8.98% FID improvement with SDE sampling

Enhanced velocity field accuracy near boundaries

Abstract

Rectified Flow offers a simple and effective approach to high-quality generative modeling by learning a velocity field. However, we identify a limitation in directly modeling the velocity with an unconstrained neural network: the learned velocity often fails to satisfy certain boundary conditions, leading to inaccurate velocity field estimations that deviate from the desired ODE. This issue is particularly critical during stochastic sampling at inference, as the score function's errors are amplified near the boundary. To mitigate this, we propose a Boundary-enforced Rectified Flow Model (Boundary RF Model), in which we enforce boundary conditions with a minimal code modification. Boundary RF Model improves performance over vanilla RF model, demonstrating 8.01% improvement in FID score on ImageNet using ODE sampling and 8.98% improvement using SDE sampling.