Difflare: Removing Image Lens Flare with Latent Diffusion Model

TL;DR

Difflare leverages pre-trained diffusion models and physical priors to effectively remove lens flare from images, achieving state-of-the-art results with improved fidelity and perceptual quality.

Contribution

Introduces Difflare, a novel lens flare removal method that utilizes pre-trained diffusion models, a structural guidance module, and luminance gradient prior for enhanced performance.

Findings

Achieves state-of-the-art lens flare removal results.

Restores images with higher fidelity and perceptual quality.

Effectively incorporates physical priors into the diffusion-based framework.

Abstract

The recovery of high-quality images from images corrupted by lens flare presents a significant challenge in low-level vision. Contemporary deep learning methods frequently entail training a lens flare removing model from scratch. However, these methods, despite their noticeable success, fail to utilize the generative prior learned by pre-trained models, resulting in unsatisfactory performance in lens flare removal. Furthermore, there are only few works considering the physical priors relevant to flare removal. To address these issues, we introduce Difflare, a novel approach designed for lens flare removal. To leverage the generative prior learned by Pre-Trained Diffusion Models (PTDM), we introduce a trainable Structural Guidance Injection Module (SGIM) aimed at guiding the restoration process with PTDM. Towards more efficient training, we employ Difflare in the latent space. To address information loss resulting from latent compression and the stochastic sampling process of PTDM, we introduce an Adaptive Feature Fusion Module (AFFM), which incorporates the Luminance Gradient Prior (LGP) of lens flare to dynamically regulate feature extraction. Extensive experiments demonstrate that our proposed Difflare achieves state-of-the-art performance in real-world lens flare removal, restoring images corrupted by flare with improved fidelity and perceptual quality. The codes will be released soon.