CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance

TL;DR

This paper introduces CFG-Ctrl, a control-based framework for classifier-free diffusion guidance, addressing stability issues in existing methods by applying sliding mode control, leading to improved semantic alignment and robustness in text-to-image generation.

Contribution

The paper proposes SMC-CFG, a novel nonlinear control approach using sliding mode control to enhance stability and performance of diffusion guidance methods.

Findings

SMC-CFG outperforms standard CFG in semantic alignment.

Enhanced robustness across various guidance scales.

Theoretical stability analysis supports finite-time convergence.

Abstract

Classifier-Free Guidance (CFG) has emerged as a central approach for enhancing semantic alignment in flow-based diffusion models. In this paper, we explore a unified framework called CFG-Ctrl, which reinterprets CFG as a control applied to the first-order continuous-time generative flow, using the conditional-unconditional discrepancy as an error signal to adjust the velocity field. From this perspective, we summarize vanilla CFG as a proportional controller (P-control) with fixed gain, and typical follow-up variants develop extended control-law designs derived from it. However, existing methods mainly rely on linear control, inherently leading to instability, overshooting, and degraded semantic fidelity especially on large guidance scales. To address this, we introduce Sliding Mode Control CFG (SMC-CFG), which enforces the generative flow toward a rapidly convergent sliding manifold. Specifically, we define an exponential sliding mode surface over the semantic prediction error and introduce a switching control term to establish nonlinear feedback-guided correction. Moreover, we provide a Lyapunov stability analysis to theoretically support finite-time convergence. Experiments across text-to-image generation models including Stable Diffusion 3.5, Flux, and Qwen-Image demonstrate that SMC-CFG outperforms standard CFG in semantic alignment and enhances robustness across a wide range of guidance scales. Project Page: https://hanyang-21.github.io/CFG-Ctrl