Multiple weather images restoration using the task transformer and adaptive mixup strategy

TL;DR

This paper presents a multi-task weather removal model that adaptively handles complex mixed weather conditions in images, utilizing a task sequence generator, Fourier convolution, and adaptive upsampling to improve autonomous driving image clarity.

Contribution

The paper introduces a novel multi-task severe weather removal model with a weather task sequence generator and adaptive strategies, addressing the challenge of mixed weather conditions in images.

Findings

Achieved state-of-the-art performance on a public weather removal dataset.

Effectively handles complex and mixed weather conditions in images.

Utilizes Fourier convolution and adaptive upsampling for improved feature processing.

Abstract

The current state-of-the-art in severe weather removal predominantly focuses on single-task applications, such as rain removal, haze removal, and snow removal. However, real-world weather conditions often consist of a mixture of several weather types, and the degree of weather mixing in autonomous driving scenarios remains unknown. In the presence of complex and diverse weather conditions, a single weather removal model often encounters challenges in producing clear images from severe weather images. Therefore, there is a need for the development of multi-task severe weather removal models that can effectively handle mixed weather conditions and improve image quality in autonomous driving scenarios. In this paper, we introduce a novel multi-task severe weather removal model that can effectively handle complex weather conditions in an adaptive manner. Our model incorporates a weather task sequence generator, enabling the self-attention mechanism to selectively focus on features specific to different weather types. To tackle the challenge of repairing large areas of weather degradation, we introduce Fast Fourier Convolution (FFC) to increase the receptive field. Additionally, we propose an adaptive upsampling technique that effectively processes both the weather task information and underlying image features by selectively retaining relevant information. Our proposed model has achieved state-of-the-art performance on the publicly available dataset.