On tuning consistent annealed sampling for denoising score matching

TL;DR

This paper analyzes and improves the hyper-parameter tuning of consistent annealed sampling in score-based generative models, enhancing sampling efficiency and flexibility for image and audio synthesis.

Contribution

It provides a detailed overview of sampling schemes, studies hyper-parameter boundaries, and proposes a formulation that simplifies tuning with fewer or variable steps.

Findings

Hyper-parameter boundaries for consistent annealed sampling are characterized.

A new formulation facilitates hyper-parameter tuning with fewer steps.

Connections between sampling schemes are elucidated.

Abstract

Score-based generative models provide state-of-the-art quality for image and audio synthesis. Sampling from these models is performed iteratively, typically employing a discretized series of noise levels and a predefined scheme. In this note, we first overview three common sampling schemes for models trained with denoising score matching. Next, we focus on one of them, consistent annealed sampling, and study its hyper-parameter boundaries. We then highlight a possible formulation of such hyper-parameter that explicitly considers those boundaries and facilitates tuning when using few or a variable number of steps. Finally, we highlight some connections of the formulation with other sampling schemes.