Navigating the Exploration-Exploitation Tradeoff in Inference-Time Scaling of Diffusion Models

TL;DR

This paper introduces novel strategies to improve inference-time scaling of diffusion models by balancing exploration and exploitation, leading to higher quality samples without extra computational cost.

Contribution

It proposes Funnel Schedule and Adaptive Temperature methods tailored for diffusion models, addressing the exploration-exploitation dilemma during inference.

Findings

Enhanced sample quality on multiple benchmarks

Outperforms previous inference scaling methods

Effective in state-of-the-art text-to-image diffusion models

Abstract

Inference-time scaling has achieved remarkable success in language models, yet its adaptation to diffusion models remains underexplored. We observe that the efficacy of recent Sequential Monte Carlo (SMC)-based methods largely stems from globally fitting the The reward-tilted distribution, which inherently preserves diversity during multi-modal search. However, current applications of SMC to diffusion models face a fundamental dilemma: early-stage noise samples offer high potential for improvement but are difficult to evaluate accurately, whereas late-stage samples can be reliably assessed but are largely irreversible. To address this exploration-exploitation trade-off, we approach the problem from the perspective of the search algorithm and propose two strategies: Funnel Schedule and Adaptive Temperature. These simple yet effective methods are tailored to the unique generation dynamics and phase-transition behavior of diffusion models. By progressively reducing the number of maintained particles and down-weighting the influence of early-stage rewards, our methods significantly enhance sample quality without increasing the total number of Noise Function Evaluations. Experimental results on multiple benchmarks and state-of-the-art text-to-image diffusion models demonstrate that our approach outperforms previous baselines.