Improved Vector Quantized Diffusion Models

TL;DR

This paper enhances vector quantized diffusion models for text-to-image synthesis by introducing improved sampling strategies, significantly boosting image quality and FID scores across datasets.

Contribution

It proposes a more effective classifier-free guidance method and a high-quality inference strategy to improve VQ-Diffusion's sample quality.

Findings

Achieved an 8.44 FID on MSCOCO, surpassing previous models.

Reduced FID from 11.89 to 4.83 on ImageNet.

Demonstrated large margins of improvement over vanilla VQ-Diffusion.

Abstract

Vector quantized diffusion (VQ-Diffusion) is a powerful generative model for text-to-image synthesis, but sometimes can still generate low-quality samples or weakly correlated images with text input. We find these issues are mainly due to the flawed sampling strategy. In this paper, we propose two important techniques to further improve the sample quality of VQ-Diffusion. 1) We explore classifier-free guidance sampling for discrete denoising diffusion model and propose a more general and effective implementation of classifier-free guidance. 2) We present a high-quality inference strategy to alleviate the joint distribution issue in VQ-Diffusion. Finally, we conduct experiments on various datasets to validate their effectiveness and show that the improved VQ-Diffusion suppresses the vanilla version by large margins. We achieve an 8.44 FID score on MSCOCO, surpassing VQ-Diffusion by 5.42 FID score. When trained on ImageNet, we dramatically improve the FID score from 11.89 to 4.83, demonstrating the superiority of our proposed techniques.