Unveiling the Underwater World: CLIP Perception Model-Guided Underwater Image Enhancement

TL;DR

This paper introduces a novel underwater image enhancement method guided by a CLIP perception model, improving perceptual quality by aligning enhancement with human visual perception and employing curriculum contrastive regularization.

Contribution

The work integrates CLIP-based perception modeling into underwater image enhancement, introducing a perception loss and regularization that better align with human perception and improve image quality.

Findings

Outperforms state-of-the-art methods in visual quality

Enhances generalization ability across diverse underwater images

Effectively balances enhancement levels to avoid under- or over-enhancement

Abstract

High-quality underwater images are essential for both machine vision tasks and viewers with their aesthetic appeal.However, the quality of underwater images is severely affected by light absorption and scattering. Deep learning-based methods for Underwater Image Enhancement (UIE) have achieved good performance. However, these methods often overlook considering human perception and lack sufficient constraints within the solution space. Consequently, the enhanced images often suffer from diminished perceptual quality or poor content restoration.To address these issues, we propose a UIE method with a Contrastive Language-Image Pre-Training (CLIP) perception loss module and curriculum contrastive regularization. Above all, to develop a perception model for underwater images that more aligns with human visual perception, the visual semantic feature extraction capability of the CLIP model is leveraged to learn an appropriate prompt pair to map and evaluate the quality of underwater images. This CLIP perception model is then incorporated as a perception loss module into the enhancement network to improve the perceptual quality of enhanced images. Furthermore, the CLIP perception model is integrated with the curriculum contrastive regularization to enhance the constraints imposed on the enhanced images within the CLIP perceptual space, mitigating the risk of both under-enhancement and over-enhancement. Specifically, the CLIP perception model is employed to assess and categorize the learning difficulty level of negatives in the regularization process, ensuring comprehensive and nuanced utilization of distorted images and negatives with varied quality levels. Extensive experiments demonstrate that our method outperforms state-of-the-art methods in terms of visual quality and generalization ability.