Distribution Matching Distillation without Fake Score Network

TL;DR

This paper introduces FSF-DMD, a novel distribution matching distillation method that eliminates the need for an auxiliary fake-score network by leveraging the generator’s pseudo-velocity, improving flow-map training efficiency.

Contribution

It proposes a fake-score-network-free formulation of distribution matching distillation for flow-map generators, simplifying training and maintaining effectiveness.

Findings

FSF-DMD improves flow-map baselines on ImageNet-1K.

It achieves lower FID scores than previous DMD2 methods.

Effective even when trained from scratch or with flow-matching initialization.

Abstract

Distribution Matching Distillation (DMD) provides an effective distribution-level correction for few-step generation, while relying on an auxiliary fake-score network to track the evolving generative distribution. Recent work combines DMD-style objectives with flow-map generators to exploit both forward-divergence training and reverse-divergence correction. The fake-score estimator remains an additional component with memory and update overhead. In this work, we study whether this explicit tracker can be avoided when the generator itself has a flow-map structure. We propose Fake-Score-network-Free DMD (FSF-DMD), a DMD formulation for flow-map generators that replaces the auxiliary fake-score estimator with a generator-induced pseudo-velocity surrogate. The key observation is that the endpoint pseudo-velocity of a flow-map generator provides a tractable proxy for fake-velocity estimation, allowing the generator itself to supply the reverse-divergence signal. Building on this observation, we derive a practical objective, extend it with flow-map-consistent backward simulation, and introduce a self-teacher variant for training from scratch. In our ImageNet-1K $256 \times 256$ experiments, FSF-DMD improves flow-map baselines, reaches lower FID than the listed DMD2 comparisons in the flow-map-initialized setting, and remains effective under flow-matching initialization and training from scratch.