Diffusion Model with Perceptual Loss

TL;DR

This paper investigates why diffusion models without guidance produce unrealistic samples, revealing that the loss function choice is crucial, and introduces a perceptual loss to improve sample quality without guidance.

Contribution

The study identifies the loss objective as key to diffusion model performance and proposes a novel perceptual loss to generate realistic samples without guidance.

Findings

Perceptual loss improves sample realism

Guidance effects can be explained by perceptual supervision

Self-perceptual loss reduces reliance on guidance

Abstract

Diffusion models without guidance generate very unrealistic samples. Guidance is used ubiquitously, and previous research has attributed its effect to low-temperature sampling that improves quality by trading off diversity. However, this perspective is incomplete. Our research shows that the choice of the loss objective is the underlying reason raw diffusion models fail to generate desirable samples. In this paper, (1) our analysis shows that the loss objective plays an important role in shaping the learned distribution and the MSE loss derived from theories holds assumptions that misalign with data in practice; (2) we explain the effectiveness of guidance methods from a new perspective of perceptual supervision; (3) we validate our hypothesis by training a diffusion model with a novel self-perceptual loss objective and obtaining much more realistic samples without the need for guidance. We hope our work paves the way for future explorations of the diffusion loss objective.