Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise

TL;DR

This paper introduces Cold Diffusion, a novel class of generative models that invert arbitrary deterministic image transformations without relying on noise, challenging traditional diffusion model paradigms.

Contribution

It demonstrates that diffusion models can be generalized to invert deterministic image processes, expanding the scope beyond noise-based degradation.

Findings

Deterministic degradations can be used for generative modeling.

Diffusion models are not limited to noise-based processes.

The approach broadens the understanding of diffusion model capabilities.

Abstract

Standard diffusion models involve an image transform -- adding Gaussian noise -- and an image restoration operator that inverts this degradation. We observe that the generative behavior of diffusion models is not strongly dependent on the choice of image degradation, and in fact an entire family of generative models can be constructed by varying this choice. Even when using completely deterministic degradations (e.g., blur, masking, and more), the training and test-time update rules that underlie diffusion models can be easily generalized to create generative models. The success of these fully deterministic models calls into question the community's understanding of diffusion models, which relies on noise in either gradient Langevin dynamics or variational inference, and paves the way for generalized diffusion models that invert arbitrary processes. Our code is available at https://github.com/arpitbansal297/Cold-Diffusion-Models