ClusterDDPM: An EM clustering framework with Denoising Diffusion Probabilistic Models

TL;DR

This paper introduces ClusterDDPM, a novel EM clustering framework leveraging denoising diffusion probabilistic models to improve clustering, latent representation learning, and unsupervised generation, surpassing traditional VAE and GAN methods.

Contribution

The study proposes an innovative EM framework with DDPMs for clustering, including a theoretical analysis and demonstrating superior performance over existing models.

Findings

Enhanced clustering accuracy demonstrated in experiments

Improved latent representation quality

Superior unsupervised conditional generation results

Abstract

Variational autoencoder (VAE) and generative adversarial networks (GAN) have found widespread applications in clustering and have achieved significant success. However, the potential of these approaches may be limited due to VAE's mediocre generation capability or GAN's well-known instability during adversarial training. In contrast, denoising diffusion probabilistic models (DDPMs) represent a new and promising class of generative models that may unlock fresh dimensions in clustering. In this study, we introduce an innovative expectation-maximization (EM) framework for clustering using DDPMs. In the E-step, we aim to derive a mixture of Gaussian priors for the subsequent M-step. In the M-step, our focus lies in learning clustering-friendly latent representations for the data by employing the conditional DDPM and matching the distribution of latent representations to the mixture of Gaussian priors. We present a rigorous theoretical analysis of the optimization process in the M-step, proving that the optimizations are equivalent to maximizing the lower bound of the Q function within the vanilla EM framework under certain constraints. Comprehensive experiments validate the advantages of the proposed framework, showcasing superior performance in clustering, unsupervised conditional generation and latent representation learning.