Condition Errors Refinement in Autoregressive Image Generation with Diffusion Loss

TL;DR

This paper provides a theoretical analysis and a novel condition refinement method for autoregressive image generation with diffusion loss, demonstrating improved stability and condition consistency over existing models.

Contribution

It introduces a theoretical comparison of diffusion and autoregressive models with diffusion loss and proposes a Wasserstein Gradient Flow-based condition refinement approach.

Findings

Autoregressive models with diffusion loss effectively mitigate condition errors.

The proposed OT-based refinement ensures convergence to the ideal condition distribution.

Experiments show the method outperforms existing diffusion and autoregressive models.

Abstract

Recent studies have explored autoregressive models for image generation, with promising results, and have combined diffusion models with autoregressive frameworks to optimize image generation via diffusion losses. In this study, we present a theoretical analysis of diffusion and autoregressive models with diffusion loss, highlighting the latter's advantages. We present a theoretical comparison of conditional diffusion and autoregressive diffusion with diffusion loss, demonstrating that patch denoising optimization in autoregressive models effectively mitigates condition errors and leads to a stable condition distribution. Our analysis also reveals that autoregressive condition generation refines the condition, causing the condition error influence to decay exponentially. In addition, we introduce a novel condition refinement approach based on Optimal Transport (OT) theory to address ``condition inconsistency''. We theoretically demonstrate that formulating condition refinement as a Wasserstein Gradient Flow ensures convergence toward the ideal condition distribution, effectively mitigating condition inconsistency. Experiments demonstrate the superiority of our method over diffusion and autoregressive models with diffusion loss methods.