Bilevel Layer-Positioning LoRA for Real Image Dehazing

TL;DR

This paper introduces a novel unsupervised cross-modal guidance method using CLIP for real image dehazing, combined with a bilevel layer-positioning LoRA strategy for targeted model adaptation, achieving superior results on real-world benchmarks.

Contribution

It proposes a new haze-to-clear text-directed loss leveraging CLIP and a bilevel layer-positioning LoRA strategy for efficient, targeted model adaptation in real image dehazing.

Findings

Outperforms state-of-the-art dehazing methods on multiple benchmarks.

Effectively uses CLIP for semantic guidance without reference images.

Enables targeted layer adaptation with reduced fine-tuning costs.

Abstract

Learning-based real image dehazing methods have achieved notable progress, yet they still face adaptation challenges in diverse real haze scenes. These challenges mainly stem from the lack of effective unsupervised mechanisms for unlabeled data and the heavy cost of full model fine-tuning. To address these challenges, we propose the haze-to-clear text-directed loss that leverages CLIP's cross-modal capabilities to reformulate real image dehazing as a semantic alignment problem in latent space, thereby providing explicit unsupervised cross-modal guidance in the absence of reference images. Furthermore, we introduce the Bilevel Layer-positioning LoRA (BiLaLoRA) strategy, which learns both the LoRA parameters and automatically search the injection layers, enabling targeted adaptation of critical network layers. Extensive experiments demonstrate our superiority against state-of-the-art methods on multiple real-world dehazing benchmarks. The code is publicly available at https://github.com/YanZhang-zy/BiLaLoRA.