Language Prompt vs. Image Enhancement: Boosting Object Detection With CLIP in Hazy Environments

TL;DR

This paper introduces a novel CLIP-guided semantic enhancement method using language prompts to improve object detection in hazy environments without relying on traditional image enhancement techniques.

Contribution

It proposes CLIP-CE and FAME, innovative techniques that leverage language prompts and adaptive weighting to enhance semantics and boost detection accuracy in hazy conditions.

Findings

Achieves state-of-the-art detection performance on HazyCOCO dataset.

Demonstrates effectiveness of language prompts in semantic enhancement.

Provides a large-scale synthetic hazy dataset for benchmarking.

Abstract

Object detection in hazy environments is challenging because degraded objects are nearly invisible and their semantics are weakened by environmental noise, making it difficult for detectors to identify. Common approaches involve image enhancement to boost weakened semantics, but these methods are limited by the instability of enhanced modules. This paper proposes a novel solution by employing language prompts to enhance weakened semantics without image enhancement. Specifically, we design Approximation of Mutual Exclusion (AME) to provide credible weights for Cross-Entropy Loss, resulting in CLIP-guided Cross-Entropy Loss (CLIP-CE). The provided weights assess the semantic weakening of objects. Through the backpropagation of CLIP-CE, weakened semantics are enhanced, making degraded objects easier to detect. In addition, we present Fine-tuned AME (FAME) which adaptively fine-tunes the weight of AME based on the predicted confidence. The proposed FAME compensates for the imbalanced optimization in AME. Furthermore, we present HazyCOCO, a large-scale synthetic hazy dataset comprising 61258 images. Experimental results demonstrate that our method achieves state-of-the-art performance. The code and dataset will be released.