Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Sampling

TL;DR

This paper introduces HSO, a hierarchical schedule optimizer that significantly accelerates diffusion model sampling with minimal computational cost, achieving state-of-the-art results in low-NFE scenarios.

Contribution

The paper presents a novel bi-level optimization framework, HSO, incorporating the Midpoint Error Proxy and Spacing-Penalized Fitness for effective, robust, and efficient schedule optimization.

Findings

HSO achieves a FID of 11.94 with NFE=5 on LAION-Aesthetics.

HSO outperforms existing methods in low-NFE regimes.

Optimization takes less than 8 seconds without retraining.

Abstract

Diffusion probabilistic models have set a new standard for generative fidelity but are hindered by a slow iterative sampling process. A powerful training-free strategy to accelerate this process is Schedule Optimization, which aims to find an optimal distribution of timesteps for a fixed and small Number of Function Evaluations (NFE) to maximize sample quality. To this end, a successful schedule optimization method must adhere to four core principles: effectiveness, adaptivity, practical robustness, and computational efficiency. However, existing paradigms struggle to satisfy these principles simultaneously, motivating the need for a more advanced solution. To overcome these limitations, we propose the Hierarchical-Schedule-Optimizer (HSO), a novel and efficient bi-level optimization framework. HSO reframes the search for a globally optimal schedule into a more tractable problem by iteratively alternating between two synergistic levels: an upper-level global search for an optimal initialization strategy and a lower-level local optimization for schedule refinement. This process is guided by two key innovations: the Midpoint Error Proxy (MEP), a solver-agnostic and numerically stable objective for effective local optimization, and the Spacing-Penalized Fitness (SPF) function, which ensures practical robustness by penalizing pathologically close timesteps. Extensive experiments show that HSO sets a new state-of-the-art for training-free sampling in the extremely low-NFE regime. For instance, with an NFE of just 5, HSO achieves a remarkable FID of 11.94 on LAION-Aesthetics with Stable Diffusion v2.1. Crucially, this level of performance is attained not through costly retraining, but with a one-time optimization cost of less than 8 seconds, presenting a highly practical and efficient paradigm for diffusion model acceleration.