ProtoDiffusion: Classifier-Free Diffusion Guidance with Prototype Learning

TL;DR

ProtoDiffusion introduces learned class prototypes into diffusion models, significantly reducing training time and enhancing early-stage performance while maintaining high generation quality.

Contribution

It proposes a novel method that replaces random class embeddings with learned prototypes, accelerating training and improving early performance of diffusion models.

Findings

ProtoDiffusion outperforms baseline in early training stages.

Achieves higher quality generation faster across multiple datasets.

Reduces training time without compromising output quality.

Abstract

Diffusion models are generative models that have shown significant advantages compared to other generative models in terms of higher generation quality and more stable training. However, the computational need for training diffusion models is considerably increased. In this work, we incorporate prototype learning into diffusion models to achieve high generation quality faster than the original diffusion model. Instead of randomly initialized class embeddings, we use separately learned class prototypes as the conditioning information to guide the diffusion process. We observe that our method, called ProtoDiffusion, achieves better performance in the early stages of training compared to the baseline method, signifying that using the learned prototypes shortens the training time. We demonstrate the performance of ProtoDiffusion using various datasets and experimental settings, achieving the best performance in shorter times across all settings.