Automated Learning of Semantic Embedding Representations for Diffusion Models

TL;DR

This paper introduces a multi-level denoising autoencoder framework for diffusion models, enhancing their representation learning to produce semantically rich embeddings that outperform existing self-supervised methods.

Contribution

The work develops a novel diffusion autoencoder with sequentially consistent transformers and a timestep-dependent encoder for improved semantic embedding learning in diffusion models.

Findings

Embeddings learned by the proposed method outperform state-of-the-art self-supervised methods.

The approach achieves high discriminative semantic representation quality.

Experiments validate the effectiveness across various datasets.

Abstract

Generative models capture the true distribution of data, yielding semantically rich representations. Denoising diffusion models (DDMs) exhibit superior generative capabilities, though efficient representation learning for them are lacking. In this work, we employ a multi-level denoising autoencoder framework to expand the representation capacity of DDMs, which introduces sequentially consistent Diffusion Transformers and an additional timestep-dependent encoder to acquire embedding representations on the denoising Markov chain through self-conditional diffusion learning. Intuitively, the encoder, conditioned on the entire diffusion process, compresses high-dimensional data into directional vectors in latent under different noise levels, facilitating the learning of image embeddings across all timesteps. To verify the semantic adequacy of embeddings generated through this approach, extensive experiments are conducted on various datasets, demonstrating that optimally learned embeddings by DDMs surpass state-of-the-art self-supervised representation learning methods in most cases, achieving remarkable discriminative semantic representation quality. Our work justifies that DDMs are not only suitable for generative tasks, but also potentially advantageous for general-purpose deep learning applications.