PartDiff: Image Super-resolution with Partial Diffusion Models

TL;DR

PartDiff introduces a partial diffusion approach for image super-resolution that reduces computational costs by starting denoising from an intermediate latent state, maintaining high quality in MRI and natural images.

Contribution

The paper proposes Partial Diffusion Models that diffuse to an intermediate state, enabling fewer denoising steps and faster super-resolution without quality loss.

Findings

Significantly reduces denoising steps compared to traditional diffusion models.

Maintains high-quality super-resolution results on MRI and natural images.

Introduces latent alignment to improve approximation accuracy.

Abstract

Denoising diffusion probabilistic models (DDPMs) have achieved impressive performance on various image generation tasks, including image super-resolution. By learning to reverse the process of gradually diffusing the data distribution into Gaussian noise, DDPMs generate new data by iteratively denoising from random noise. Despite their impressive performance, diffusion-based generative models suffer from high computational costs due to the large number of denoising steps.In this paper, we first observed that the intermediate latent states gradually converge and become indistinguishable when diffusing a pair of low- and high-resolution images. This observation inspired us to propose the Partial Diffusion Model (PartDiff), which diffuses the image to an intermediate latent state instead of pure random noise, where the intermediate latent state is approximated by the latent of diffusing the low-resolution image. During generation, Partial Diffusion Models start denoising from the intermediate distribution and perform only a part of the denoising steps. Additionally, to mitigate the error caused by the approximation, we introduce "latent alignment", which aligns the latent between low- and high-resolution images during training. Experiments on both magnetic resonance imaging (MRI) and natural images show that, compared to plain diffusion-based super-resolution methods, Partial Diffusion Models significantly reduce the number of denoising steps without sacrificing the quality of generation.