Diffusion Models already have a Semantic Latent Space

TL;DR

This paper introduces a novel method called Asyrp that discovers a semantic latent space within pretrained diffusion models, enabling effective semantic image manipulation and improving editing versatility and image quality.

Contribution

The paper proposes Asyrp, a new asymmetric reverse process that uncovers a semantic latent space in frozen diffusion models, facilitating controllable and high-quality image editing.

Findings

Semantic latent space exhibits homogeneity, linearity, robustness, and consistency.

Applicable to various diffusion architectures and datasets.

Enhances image editing capabilities and quality in diffusion models.

Abstract

Diffusion models achieve outstanding generative performance in various domains. Despite their great success, they lack semantic latent space which is essential for controlling the generative process. To address the problem, we propose asymmetric reverse process (Asyrp) which discovers the semantic latent space in frozen pretrained diffusion models. Our semantic latent space, named h-space, has nice properties for accommodating semantic image manipulation: homogeneity, linearity, robustness, and consistency across timesteps. In addition, we introduce a principled design of the generative process for versatile editing and quality boost ing by quantifiable measures: editing strength of an interval and quality deficiency at a timestep. Our method is applicable to various architectures (DDPM++, iD- DPM, and ADM) and datasets (CelebA-HQ, AFHQ-dog, LSUN-church, LSUN- bedroom, and METFACES). Project page: https://kwonminki.github.io/Asyrp/