Efficient Controllable Diffusion via Optimal Classifier Guidance

TL;DR

This paper introduces SLCD, a supervised learning approach for controllable diffusion that efficiently guides sample generation towards desired objectives without complex reinforcement learning, with proven convergence and high-quality results.

Contribution

SLCD offers a simple, classifier-based method for controllable diffusion that converges to optimal solutions and matches the inference speed of base models.

Findings

SLCD achieves high-quality sample generation in image and biological sequence tasks.

The method converges to the optimal KL-regularized solution under theoretical analysis.

SLCD operates with nearly the same inference time as the base diffusion models.

Abstract

The controllable generation of diffusion models aims to steer the model to generate samples that optimize some given objective functions. It is desirable for a variety of applications including image generation, molecule generation, and DNA/sequence generation. Reinforcement Learning (RL) based fine-tuning of the base model is a popular approach but it can overfit the reward function while requiring significant resources. We frame controllable generation as a problem of finding a distribution that optimizes a KL-regularized objective function. We present SLCD -- Supervised Learning based Controllable Diffusion, which iteratively generates online data and trains a small classifier to guide the generation of the diffusion model. Similar to the standard classifier-guided diffusion, SLCD's key computation primitive is classification and does not involve any complex concepts from RL or control. Via a reduction to no-regret online learning analysis, we show that under KL divergence, the output from SLCD provably converges to the optimal solution of the KL-regularized objective. Further, we empirically demonstrate that SLCD can generate high quality samples with nearly the same inference time as the base model in both image generation with continuous diffusion and biological sequence generation with discrete diffusion. Our code is available at https://github.com/Owen-Oertell/slcd