Image Embedding for Denoising Generative Models

TL;DR

This paper explores embedding images into the latent space of Denoising Diffusion Models, especially Denoising Diffusion Implicit Models, revealing insights into their structure and properties for image editing and manipulation.

Contribution

It introduces methods for image embedding in diffusion models and uncovers the latent space's structure and its independence from specific network implementations.

Findings

Latent space embedding enables image reconstruction from noisy inputs.

Latent representations are independent of the specific network used.

Insights into semantic trajectories and editing in diffusion models.

Abstract

Denoising Diffusion models are gaining increasing popularity in the field of generative modeling for several reasons, including the simple and stable training, the excellent generative quality, and the solid probabilistic foundation. In this article, we address the problem of {\em embedding} an image into the latent space of Denoising Diffusion Models, that is finding a suitable ``noisy'' image whose denoising results in the original image. We particularly focus on Denoising Diffusion Implicit Models due to the deterministic nature of their reverse diffusion process. As a side result of our investigation, we gain a deeper insight into the structure of the latent space of diffusion models, opening interesting perspectives on its exploration, the definition of semantic trajectories, and the manipulation/conditioning of encodings for editing purposes. A particularly interesting property highlighted by our research, which is also characteristic of this class of generative models, is the independence of the latent representation from the networks implementing the reverse diffusion process. In other words, a common seed passed to different networks (each trained on the same dataset), eventually results in identical images.