Learning Quantized Adaptive Conditions for Diffusion Models

TL;DR

This paper introduces a lightweight adaptive condition method using quantized encoding to reduce trajectory curvature in diffusion models, significantly improving image quality with minimal sampling evaluations.

Contribution

It presents a novel quantized adaptive condition approach that enhances diffusion model sampling efficiency without extra regularization, maintaining image editing capabilities.

Findings

Achieves 5.14 FID on CIFAR-10 with 6 NFE

Attains 6.91 FID on FFHQ 64x64 with 6 NFE

Reaches 3.10 FID on AFHQv2 with 6 NFE

Abstract

The curvature of ODE trajectories in diffusion models hinders their ability to generate high-quality images in a few number of function evaluations (NFE). In this paper, we propose a novel and effective approach to reduce trajectory curvature by utilizing adaptive conditions. By employing a extremely light-weight quantized encoder, our method incurs only an additional 1% of training parameters, eliminates the need for extra regularization terms, yet achieves significantly better sample quality. Our approach accelerates ODE sampling while preserving the downstream task image editing capabilities of SDE techniques. Extensive experiments verify that our method can generate high quality results under extremely limited sampling costs. With only 6 NFE, we achieve 5.14 FID on CIFAR-10, 6.91 FID on FFHQ 64x64 and 3.10 FID on AFHQv2.