CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation

TL;DR

This paper investigates the linear relationship between initial noise perturbations and generated outputs in diffusion models, proposing a novel controllable sampling method that enhances control over generated data while maintaining quality.

Contribution

It reveals a linearity property in diffusion models' sampling process and introduces CCS, a new method for controllable, constrained sampling with theoretical and empirical validation.

Findings

CCS achieves more precise control over sampling.

CCS maintains high sample quality and diversity.

The linear relationship between noise and output is theoretically justified.

Abstract

Diffusion models have emerged as powerful tools for generative tasks, producing high-quality outputs across diverse domains. However, how the generated data responds to the initial noise perturbation in diffusion models remains under-explored, which hinders understanding the controllability of the sampling process. In this work, we first observe an interesting phenomenon: the relationship between the change of generation outputs and the scale of initial noise perturbation is highly linear through the diffusion ODE sampling. Then we provide both theoretical and empirical study to justify this linearity property of this input-output (noise-generation data) relationship. Inspired by these new insights, we propose a novel Controllable and Constrained Sampling method (CCS) together with a new controller algorithm for diffusion models to sample with desired statistical properties while preserving good sample quality. We perform extensive experiments to compare our proposed sampling approach with other methods on both sampling controllability and sampled data quality. Results show that our CCS method achieves more precisely controlled sampling while maintaining superior sample quality and diversity.