CoDe: Blockwise Control for Denoising Diffusion Models

TL;DR

This paper introduces CoDe, a simple gradient-free inference-time guidance method for diffusion models that improves alignment with downstream rewards without requiring model finetuning or differentiable guidance functions.

Contribution

The paper proposes CoDe, a novel blockwise, gradient-free guidance technique for diffusion models that enhances downstream task alignment during inference.

Findings

CoDe achieves competitive reward alignment performance.

It offers a favorable trade-off between accuracy and inference cost.

CoDe simplifies the guidance process without model finetuning.

Abstract

Aligning diffusion models to downstream tasks often requires finetuning new models or gradient-based guidance at inference time to enable sampling from the reward-tilted posterior. In this work, we explore a simple inference-time gradient-free guidance approach, called controlled denoising (CoDe), that circumvents the need for differentiable guidance functions and model finetuning. CoDe is a blockwise sampling method applied during intermediate denoising steps, allowing for alignment with downstream rewards. Our experiments demonstrate that, despite its simplicity, CoDe offers a favorable trade-off between reward alignment, prompt instruction following, and inference cost, achieving a competitive performance against the state-of-the-art baselines. Our code is available at: https://github.com/anujinho/code.