Can We Change the Stroke Size for Easier Diffusion?

TL;DR

This paper investigates stroke-size control in diffusion models to improve their performance in low signal-to-noise scenarios by adjusting the effective roughness of predictions and targets.

Contribution

It introduces stroke-size control as a novel intervention to mitigate low signal-to-noise challenges in diffusion models.

Findings

Stroke-size control alters the effective roughness of predictions.

Adjusting stroke size can ease the low signal-to-noise challenge.

The approach provides a new way to improve diffusion model robustness.

Abstract

Diffusion models can be challenged in the low signal-to-noise regime, where they have to make pixel-level predictions despite the presence of high noise. The geometric intuition is akin to using the finest stroke for oil painting throughout, which may be ineffective. We therefore study stroke-size control as a controlled intervention that changes the effective roughness of the supervised target, predictions and perturbations across timesteps, in an attempt to ease the low signal-to-noise challenge.