Efficient Coarse-to-Fine Diffusion Models with Time Step Sequence Redistribution

TL;DR

This paper introduces a novel approach to diffusion models that reduces computational costs by identifying early-stage redundancies and redistributing time steps, enabling efficient high-quality image generation on resource-limited devices.

Contribution

The paper proposes Coarse-to-Fine Denoising and Time Step Sequence Redistribution techniques, significantly decreasing computation while maintaining image quality in diffusion models.

Findings

Achieves 80-90% reduction in computation on CIFAR10 and LSUN-Church datasets.

Requires less than 10 minutes for search in time step redistribution.

Maintains near-lossless image quality with reduced computational cost.

Abstract

Recently, diffusion models (DMs) have made significant strides in high-quality image generation. However, the multi-step denoising process often results in considerable computational overhead, impeding deployment on resource-constrained edge devices. Existing methods mitigate this issue by compressing models and adjusting the time step sequence. However, they overlook input redundancy and require lengthy search times. In this paper, we propose Coarse-to-Fine Diffusion Models with Time Step Sequence Redistribution. Recognizing indistinguishable early-stage generated images, we introduce Coarse-to-Fine Denoising (C2F) to reduce computation during coarse feature generation. Furthermore, we design Time Step Sequence Redistribution (TRD) for efficient sampling trajectory adjustment, requiring less than 10 minutes for search. Experimental results demonstrate that the proposed methods achieve near-lossless performance with an 80% to 90% reduction in computation on CIFAR10 and LSUN-Church.