Image Restoration via Diffusion Models with Dynamic Resolution

TL;DR

This paper introduces dynamic resolution diffusion models for image restoration, significantly reducing computational costs while maintaining high reconstruction quality, and proposes novel methods like SubDAPS and SubDAPS++.

Contribution

It presents a new framework using dynamic resolution DMs to accelerate inference in image restoration, with fine-tuning and adaptations of existing pixel-space methods.

Findings

Proposed methods outperform recent DM-based approaches in most scenarios.

SubDAPS++ achieves improved efficiency and quality over baseline methods.

Empirical results demonstrate faster inference with comparable or better restoration fidelity.

Abstract

Diffusion models (DMs) have exhibited remarkable efficacy in various image restoration tasks. However, existing approaches typically operate within the high-dimensional pixel space, resulting in high computational overhead. While methods based on latent DMs seek to alleviate this issue by utilizing the compressed latent space of a variational autoencoder, they require repeated encoder-decoder inference. This introduces significant additional computational burdens, often resulting in runtime performance that is even inferior to that of their pixel-space counterparts. To mitigate the computational inefficiency, this work proposes projecting data into lower-dimensional subspaces using dynamic resolution DMs to accelerate the inference process. We first fine-tune pre-trained DMs for dynamic resolution priors and adapt DPS and DAPS, which are two widely used pixel-space methods for general image restoration tasks, into the proposed framework, yielding methods we refer to as SubDPS and SubDAPS, respectively. Given the favorable inference speed and reconstruction fidelity of SubDAPS, we introduce an enhanced variant termed SubDAPS++ to further boost both reconstruction efficiency and quality. Empirical evaluations across diverse image datasets and various restoration tasks demonstrate that the proposed methods outperform recent DM-based approaches in the majority of experimental scenarios. The code is available at https://github.com/StarNextDay/SubDAPS.git.