CLIP-Guided Unsupervised Semantic-Aware Exposure Correction

TL;DR

This paper introduces an unsupervised, semantic-aware exposure correction method that leverages CLIP and FastSAM models to improve image quality without manual labels, effectively restoring details and colors in real-world images.

Contribution

It proposes a novel unsupervised exposure correction network integrating semantic information from FastSAM and CLIP, with a pseudo-ground truth generator guided by CLIP for automatic correction.

Findings

Outperforms state-of-the-art unsupervised methods in quantitative metrics.

Effectively restores details and colors in real-world exposure images.

Maintains semantic consistency during correction.

Abstract

Improper exposure often leads to severe loss of details, color distortion, and reduced contrast. Exposure correction still faces two critical challenges: (1) the ignorance of object-wise regional semantic information causes the color shift artifacts; (2) real-world exposure images generally have no ground-truth labels, and its labeling entails massive manual editing. To tackle the challenges, we propose a new unsupervised semantic-aware exposure correction network. It contains an adaptive semantic-aware fusion module, which effectively fuses the semantic information extracted from a pre-trained Fast Segment Anything Model into a shared image feature space. Then the fused features are used by our multi-scale residual spatial mamba group to restore the details and adjust the exposure. To avoid manual editing, we propose a pseudo-ground truth generator guided by CLIP, which is fine-tuned to automatically identify exposure situations and instruct the tailored corrections. Also, we leverage the rich priors from the FastSAM and CLIP to develop a semantic-prompt consistency loss to enforce semantic consistency and image-prompt alignment for unsupervised training. Comprehensive experimental results illustrate the effectiveness of our method in correcting real-world exposure images and outperforms state-of-the-art unsupervised methods both numerically and visually.