Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model

TL;DR

This paper introduces an energy-informed diffusion model leveraging CLIP priors and dual energy functions to improve unpaired photo-realistic image deraining, effectively removing rain streaks while preserving image quality.

Contribution

It proposes a novel energy-informed diffusion framework with dual energy functions and CLIP priors for unpaired image deraining, advancing the state-of-the-art in this task.

Findings

Outperforms existing unpaired deraining methods in quantitative metrics.

Effectively removes rain streaks while maintaining image content.

Demonstrates superior performance in both supervised and no-reference evaluations.

Abstract

Existing unpaired image deraining approaches face challenges in accurately capture the distinguishing characteristics between the rainy and clean domains, resulting in residual degradation and color distortion within the reconstructed images. To this end, we propose an energy-informed diffusion model for unpaired photo-realistic image deraining (UPID-EDM). Initially, we delve into the intricate visual-language priors embedded within the contrastive language-image pre-training model (CLIP), and demonstrate that the CLIP priors aid in the discrimination of rainy and clean images. Furthermore, we introduce a dual-consistent energy function (DEF) that retains the rain-irrelevant characteristics while eliminating the rain-relevant features. This energy function is trained by the non-corresponding rainy and clean images. In addition, we employ the rain-relevance discarding energy function (RDEF) and the rain-irrelevance preserving energy function (RPEF) to direct the reverse sampling procedure of a pre-trained diffusion model, effectively removing the rain streaks while preserving the image contents. Extensive experiments demonstrate that our energy-informed model surpasses the existing unpaired learning approaches in terms of both supervised and no-reference metrics.