Iterative Prompt Learning for Unsupervised Backlit Image Enhancement

TL;DR

This paper introduces CLIP-LIT, an unsupervised backlit image enhancement method that leverages CLIP's open-world prior through iterative prompt learning to improve image quality without paired data.

Contribution

The paper presents a novel unsupervised enhancement framework that uses iterative prompt learning with CLIP to optimize backlit image enhancement, addressing prompt accuracy challenges.

Findings

Outperforms state-of-the-art methods in visual quality.

Demonstrates strong generalization without paired training data.

Effectively distinguishes and enhances heterogeneous luminance regions.

Abstract

We propose a novel unsupervised backlit image enhancement method, abbreviated as CLIP-LIT, by exploring the potential of Contrastive Language-Image Pre-Training (CLIP) for pixel-level image enhancement. We show that the open-world CLIP prior not only aids in distinguishing between backlit and well-lit images, but also in perceiving heterogeneous regions with different luminance, facilitating the optimization of the enhancement network. Unlike high-level and image manipulation tasks, directly applying CLIP to enhancement tasks is non-trivial, owing to the difficulty in finding accurate prompts. To solve this issue, we devise a prompt learning framework that first learns an initial prompt pair by constraining the text-image similarity between the prompt (negative/positive sample) and the corresponding image (backlit image/well-lit image) in the CLIP latent space. Then, we train the enhancement network based on the text-image similarity between the enhanced result and the initial prompt pair. To further improve the accuracy of the initial prompt pair, we iteratively fine-tune the prompt learning framework to reduce the distribution gaps between the backlit images, enhanced results, and well-lit images via rank learning, boosting the enhancement performance. Our method alternates between updating the prompt learning framework and enhancement network until visually pleasing results are achieved. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in terms of visual quality and generalization ability, without requiring any paired data.