Multi-Weather Image Restoration via Histogram-Based Transformer Feature Enhancement

TL;DR

This paper introduces a novel histogram-based transformer model for multi-weather image restoration, effectively handling complex, mixed weather conditions to improve image quality in practical scenarios like autonomous driving.

Contribution

It proposes a new model with a Task Sequence Generator and Histogram Transformer backbone to address mixed weather restoration, a challenge not well handled by existing single-weather models.

Findings

Achieves state-of-the-art results on public datasets.

Effectively captures global and local dynamic range features.

Handles complex, mixed weather conditions in images.

Abstract

Currently, the mainstream restoration tasks under adverse weather conditions have predominantly focused on single-weather scenarios. However, in reality, multiple weather conditions always coexist and their degree of mixing is usually unknown. Under such complex and diverse weather conditions, single-weather restoration models struggle to meet practical demands. This is particularly critical in fields such as autonomous driving, where there is an urgent need for a model capable of effectively handling mixed weather conditions and enhancing image quality in an automated manner. In this paper, we propose a Task Sequence Generator module that, in conjunction with the Task Intra-patch Block, effectively extracts task-specific features embedded in degraded images. The Task Intra-patch Block introduces an external learnable sequence that aids the network in capturing task-specific information. Additionally, we employ a histogram-based transformer module as the backbone of our network, enabling the capture of both global and local dynamic range features. Our proposed model achieves state-of-the-art performance on public datasets.