Diffusion Sampling Path Tells More: An Efficient Plug-and-Play Strategy for Sample Filtering

TL;DR

This paper introduces CFG-Rejection, a plug-and-play method that filters low-quality samples early in the diffusion process by exploiting a correlation between sample quality and denoising trajectory characteristics, without extra training or external signals.

Contribution

It uncovers a novel link between sample quality and denoising trajectory features, enabling an efficient, model-agnostic sample filtering strategy during diffusion sampling.

Findings

Significant improvement in human preference scores.

Enhanced performance on challenging benchmarks.

No need for model retraining or external rewards.

Abstract

Diffusion models often exhibit inconsistent sample quality due to stochastic variations inherent in their sampling trajectories. Although training-based fine-tuning (e.g. DDPO [1]) and inference-time alignment techniques[2] aim to improve sample fidelity, they typically necessitate full denoising processes and external reward signals. This incurs substantial computational costs, hindering their broader applicability. In this work, we unveil an intriguing phenomenon: a previously unobserved yet exploitable link between sample quality and characteristics of the denoising trajectory during classifier-free guidance (CFG). Specifically, we identify a strong correlation between high-density regions of the sample distribution and the Accumulated Score Differences (ASD)--the cumulative divergence between conditional and unconditional scores. Leveraging this insight, we introduce CFG-Rejection, an efficient, plug-and-play strategy that filters low-quality samples at an early stage of the denoising process, crucially without requiring external reward signals or model retraining. Importantly, our approach necessitates no modifications to model architectures or sampling schedules and maintains full compatibility with existing diffusion frameworks. We validate the effectiveness of CFG-Rejection in image generation through extensive experiments, demonstrating marked improvements on human preference scores (HPSv2, PickScore) and challenging benchmarks (GenEval, DPG-Bench). We anticipate that CFG-Rejection will offer significant advantages for diverse generative modalities beyond images, paving the way for more efficient and reliable high-quality sample generation.