Unified Image Restoration and Enhancement: Degradation Calibrated Cycle Reconstruction Diffusion Model

TL;DR

CycleRDM is a unified diffusion-based framework that effectively combines image restoration and enhancement tasks through multi-domain learning, frequency domain calibration, and multimodal guidance, achieving superior results with minimal training data.

Contribution

The paper introduces CycleRDM, a novel diffusion model that unifies restoration and enhancement with frequency domain calibration and multimodal prompts, improving efficiency and quality.

Findings

Effective generalization to various tasks

High reconstruction and perceptual quality

Requires few training samples

Abstract

Image restoration and enhancement are pivotal for numerous computer vision applications, yet unifying these tasks efficiently remains a significant challenge. Inspired by the iterative refinement capabilities of diffusion models, we propose CycleRDM, a novel framework designed to unify restoration and enhancement tasks while achieving high-quality mapping. Specifically, CycleRDM first learns the mapping relationships among the degraded domain, the rough normal domain, and the normal domain through a two-stage diffusion inference process. Subsequently, we transfer the final calibration process to the wavelet low-frequency domain using discrete wavelet transform, performing fine-grained calibration from a frequency domain perspective by leveraging task-specific frequency spaces. To improve restoration quality, we design a feature gain module for the decomposed wavelet high-frequency domain to eliminate redundant features. Additionally, we employ multimodal textual prompts and Fourier transform to drive stable denoising and reduce randomness during the inference process. After extensive validation, CycleRDM can be effectively generalized to a wide range of image restoration and enhancement tasks while requiring only a small number of training samples to be significantly superior on various benchmarks of reconstruction quality and perceptual quality. The source code will be available at https://github.com/hejh8/CycleRDM.