Easing Color Shifts in Score-Based Diffusion Models

TL;DR

This paper proposes a simple architectural modification to score-based diffusion models that effectively mitigates color shifts in generated images, regardless of image size, improving image quality.

Contribution

It introduces a nonlinear bypass connection in the score network to address color shifts, demonstrating its effectiveness across different image sizes.

Findings

Significant reduction in color shifts in generated images.

Improved image quality with the proposed architecture.

Effectiveness independent of image size.

Abstract

Generated images of score-based models can suffer from errors in their spatial means, an effect, referred to as a color shift, which grows for larger images. This paper investigates a previously-introduced approach to mitigate color shifts in score-based diffusion models. We quantify the performance of a nonlinear bypass connection in the score network, designed to process the spatial mean of the input and to predict the mean of the score function. We show that this network architecture substantially improves the resulting quality of the generated images, and that this improvement is approximately independent of the size of the generated images. As a result, this modified architecture offers a simple solution for the color shift problem across image sizes. We additionally discuss the origin of color shifts in an idealized setting in order to motivate the approach.