ShiftDDPMs: Exploring Conditional Diffusion Models by Shifting Diffusion Trajectories

TL;DR

ShiftDDPMs introduce a novel approach to conditional diffusion models by incorporating conditions into the forward process and using shifting rules to allocate unique diffusion trajectories, enhancing modeling capacity.

Contribution

The paper proposes ShiftDDPMs, a flexible conditional diffusion model that disperses condition modeling across all timesteps using shifting rules in an extra latent space.

Findings

Improved image synthesis quality demonstrated through experiments.

Enhanced condition modeling capacity over existing methods.

Unified framework connecting related diffusion techniques.

Abstract

Diffusion models have recently exhibited remarkable abilities to synthesize striking image samples since the introduction of denoising diffusion probabilistic models (DDPMs). Their key idea is to disrupt images into noise through a fixed forward process and learn its reverse process to generate samples from noise in a denoising way. For conditional DDPMs, most existing practices relate conditions only to the reverse process and fit it to the reversal of unconditional forward process. We find this will limit the condition modeling and generation in a small time window. In this paper, we propose a novel and flexible conditional diffusion model by introducing conditions into the forward process. We utilize extra latent space to allocate an exclusive diffusion trajectory for each condition based on some shifting rules, which will disperse condition modeling to all timesteps and improve the learning capacity of model. We formulate our method, which we call \textbf{ShiftDDPMs}, and provide a unified point of view on existing related methods. Extensive qualitative and quantitative experiments on image synthesis demonstrate the feasibility and effectiveness of ShiftDDPMs.