REG: Rectified Gradient Guidance for Conditional Diffusion Models

TL;DR

This paper introduces Rectified Gradient Guidance (REG), a theoretically grounded method that improves conditional diffusion model generation by better approximating optimal guidance, leading to enhanced performance in image and text-to-image tasks.

Contribution

The paper proposes REG, a new guidance technique based on a valid joint distribution objective, bridging the gap between theory and practice in diffusion model guidance methods.

Findings

REG outperforms prior guidance methods in approximating the optimal solution.

Incorporating REG improves FID and CLIP scores in image generation tasks.

Experiments on 1D, 2D, and ImageNet demonstrate consistent performance gains.

Abstract

Guidance techniques are simple yet effective for improving conditional generation in diffusion models. Albeit their empirical success, the practical implementation of guidance diverges significantly from its theoretical motivation. In this paper, we reconcile this discrepancy by replacing the scaled marginal distribution target, which we prove theoretically invalid, with a valid scaled joint distribution objective. Additionally, we show that the established guidance implementations are approximations to the intractable optimal solution under no future foresight constraint. Building on these theoretical insights, we propose rectified gradient guidance (REG), a versatile enhancement designed to boost the performance of existing guidance methods. Experiments on 1D and 2D demonstrate that REG provides a better approximation to the optimal solution than prior guidance techniques, validating the proposed theoretical framework. Extensive experiments on class-conditional ImageNet and text-to-image generation tasks show that incorporating REG consistently improves FID and Inception/CLIP scores across various settings compared to its absence.