Path-Guided Flow Matching for Dataset Distillation

TL;DR

This paper introduces Path-Guided Flow Matching (PGFM), a novel flow matching framework for dataset distillation that achieves efficient, deterministic synthetic data generation with improved performance and diversity compared to diffusion-based methods.

Contribution

PGFM is the first flow matching-based approach for generative distillation, enabling fast, deterministic synthesis with a novel path-to-prototype guidance algorithm.

Findings

PGFM matches or surpasses diffusion-based distillation performance.

PGFM is 7.6 times more efficient than diffusion methods.

PGFM achieves 78% mode coverage on high-resolution benchmarks.

Abstract

Dataset distillation compresses large datasets into compact synthetic sets with comparable performance in training models. Despite recent progress on diffusion-based distillation, this type of method typically depends on heuristic guidance or prototype assignment, which comes with time-consuming sampling and trajectory instability and thus hurts downstream generalization especially under strong control or low IPC. We propose \emph{Path-Guided Flow Matching (PGFM)}, the first flow matching-based framework for generative distillation, which enables fast deterministic synthesis by solving an ODE in a few steps. PGFM conducts flow matching in the latent space of a frozen VAE to learn class-conditional transport from Gaussian noise to data distribution. Particularly, we develop a continuous path-to-prototype guidance algorithm for ODE-consistent path control, which allows trajectories to reliably land on assigned prototypes while preserving diversity and efficiency. Extensive experiments across high-resolution benchmarks demonstrate that PGFM matches or surpasses prior diffusion-based distillation approaches with fewer steps of sampling while delivering competitive performance with remarkably improved efficiency, e.g., 7.6$\times$ more efficient than the diffusion-based counterparts with 78\% mode coverage.