Blockwise Flow Matching: Improving Flow Matching Models For Efficient High-Quality Generation

TL;DR

This paper introduces Blockwise Flow Matching (BFM), a novel approach that partitions the generative process into segments with specialized models, significantly improving inference efficiency and sample quality in flow-based generative models.

Contribution

The paper proposes a new blockwise framework for flow matching models, along with semantic guidance and residual approximation techniques, enhancing efficiency and fidelity over prior methods.

Findings

Achieves 2.1x to 4.9x inference acceleration on ImageNet 256x256.

Establishes a better Pareto trade-off between speed and quality.

Demonstrates improved sample quality with specialized velocity blocks.

Abstract

Recently, Flow Matching models have pushed the boundaries of high-fidelity data generation across a wide range of domains. It typically employs a single large network to learn the entire generative trajectory from noise to data. Despite their effectiveness, this design struggles to capture distinct signal characteristics across timesteps simultaneously and incurs substantial inference costs due to the iterative evaluation of the entire model. To address these limitations, we propose Blockwise Flow Matching (BFM), a novel framework that partitions the generative trajectory into multiple temporal segments, each modeled by smaller but specialized velocity blocks. This blockwise design enables each block to specialize effectively in its designated interval, improving inference efficiency and sample quality. To further enhance generation fidelity, we introduce a Semantic Feature Guidance module that explicitly conditions velocity blocks on semantically rich features aligned with pretrained representations. Additionally, we propose a lightweight Feature Residual Approximation strategy that preserves semantic quality while significantly reducing inference cost. Extensive experiments on ImageNet 256x256 demonstrate that BFM establishes a substantially improved Pareto frontier over existing Flow Matching methods, achieving 2.1x to 4.9x accelerations in inference complexity at comparable generation performance. Code is available at https://github.com/mlvlab/BFM.