Natural Language Supervision for Low-light Image Enhancement

TL;DR

This paper introduces NaLSuper, a novel low-light image enhancement method using natural language supervision and cross-modal feature alignment, improving visual quality by leveraging textual descriptions and advanced attention mechanisms.

Contribution

The paper proposes a new framework that integrates natural language supervision with image enhancement, utilizing a textual guidance mechanism and information fusion attention to improve results.

Findings

NaLSuper outperforms existing LLIE methods in robustness and effectiveness.

The proposed TCM and IFA modules enhance feature extraction and fusion.

Extensive experiments validate the superiority of the approach.

Abstract

With the development of deep learning, numerous methods for low-light image enhancement (LLIE) have demonstrated remarkable performance. Mainstream LLIE methods typically learn an end-to-end mapping based on pairs of low-light and normal-light images. However, normal-light images under varying illumination conditions serve as reference images, making it difficult to define a ``perfect'' reference image This leads to the challenge of reconciling metric-oriented and visual-friendly results. Recently, many cross-modal studies have found that side information from other related modalities can guide visual representation learning. Based on this, we introduce a Natural Language Supervision (NLS) strategy, which learns feature maps from text corresponding to images, offering a general and flexible interface for describing an image under different illumination. However, image distributions conditioned on textual descriptions are highly multimodal, which makes training difficult. To address this issue, we design a Textual Guidance Conditioning Mechanism (TCM) that incorporates the connections between image regions and sentence words, enhancing the ability to capture fine-grained cross-modal cues for images and text. This strategy not only utilizes a wider range of supervised sources, but also provides a new paradigm for LLIE based on visual and textual feature alignment. In order to effectively identify and merge features from various levels of image and textual information, we design an Information Fusion Attention (IFA) module to enhance different regions at different levels. We integrate the proposed TCM and IFA into a Natural Language Supervision network for LLIE, named NaLSuper. Finally, extensive experiments demonstrate the robustness and superior effectiveness of our proposed NaLSuper.