Distance Marching for Generative Modeling

TL;DR

Distance Marching introduces a novel time-unconditional generative modeling approach that improves image quality and efficiency by focusing on closer targets and better denoising directions, outperforming recent baselines.

Contribution

The paper proposes Distance Marching, a new method for time-unconditional generative modeling that enhances denoising accuracy and sampling efficiency using distance field-inspired losses and inference techniques.

Findings

Improves FID by 13.5% on CIFAR-10 and ImageNet.

Surpasses flow matching in class-conditional ImageNet generation.

Reduces sampling steps by 40% while maintaining lower FID.

Abstract

Time-unconditional generative models learn time-independent denoising vector fields. But without time conditioning, the same noisy input may correspond to multiple noise levels and different denoising directions, which interferes with the supervision signal. Inspired by distance field modeling, we propose Distance Marching, a new time-unconditional approach with two principled inference methods. Crucially, we design losses that focus on closer targets. This yields denoising directions better directed toward the data manifold. Across architectures, Distance Marching consistently improves FID by 13.5% on CIFAR-10 and ImageNet over recent time-unconditional baselines. For class-conditional ImageNet generation, despite removing time input, Distance Marching surpasses flow matching using our losses and inference methods. It achieves lower FID than flow matching's final performance using 60% of the sampling steps and 13.6% lower FID on average across backbone sizes. Moreover, our distance prediction is also helpful for early stopping during sampling and for OOD detection. We hope distance field modeling can serve as a principled lens for generative modeling.