Resolution Chromatography of Diffusion Models

TL;DR

This paper introduces 'resolution chromatography' to mathematically analyze the resolution evolution in diffusion models, enabling better understanding and manipulation of image generation processes.

Contribution

It proposes a novel concept called resolution chromatography to explain and control the resolution dynamics in diffusion-based image generation.

Findings

Resolution chromatography accurately predicts dominant resolution levels at each time step.

The method enables effective upscaling of pre-trained models to higher resolutions.

It facilitates time-dependent prompt composition for improved image synthesis.

Abstract

Diffusion models generate high-resolution images through iterative stochastic processes. In particular, the denoising method is one of the most popular approaches that predicts the noise in samples and denoises it at each time step. It has been commonly observed that the resolution of generated samples changes over time, starting off blurry and coarse, and becoming sharper and finer. In this paper, we introduce "resolution chromatography" that indicates the signal generation rate of each resolution, which is very helpful concept to mathematically explain this coarse-to-fine behavior in generation process, to understand the role of noise schedule, and to design time-dependent modulation. Using resolution chromatography, we determine which resolution level becomes dominant at a specific time step, and experimentally verify our theory with text-to-image diffusion models. We also propose some direct applications utilizing the concept: upscaling pre-trained models to higher resolutions and time-dependent prompt composing. Our theory not only enables a better understanding of numerous pre-existing techniques for manipulating image generation, but also suggests the potential for designing better noise schedules.