End-to-end Inception-Unet based Generative Adversarial Networks for Snow and Rain Removals

TL;DR

This paper presents a dual-GAN framework with U-net architecture for effective removal of snow and rain from images, addressing challenges of particle variability and handling both degradations simultaneously.

Contribution

Introduces a novel dual-GAN approach with integrated feature extraction and U-net for improved snow and rain removal, along with a new realistic dataset for evaluation.

Findings

Significant improvement over state-of-the-art methods.

Effective handling of severe particle variations.

Validated on both synthetic and real datasets.

Abstract

The superior performance introduced by deep learning approaches in removing atmospheric particles such as snow and rain from a single image; favors their usage over classical ones. However, deep learning-based approaches still suffer from challenges related to the particle appearance characteristics such as size, type, and transparency. Furthermore, due to the unique characteristics of rain and snow particles, single network based deep learning approaches struggle in handling both degradation scenarios simultaneously. In this paper, a global framework that consists of two Generative Adversarial Networks (GANs) is proposed where each handles the removal of each particle individually. The architectures of both desnowing and deraining GANs introduce the integration of a feature extraction phase with the classical U-net generator network which in turn enhances the removal performance in the presence of severe variations in size and appearance. Furthermore, a realistic dataset that contains pairs of snowy images next to their groundtruth images estimated using a low-rank approximation approach; is presented. The experiments show that the proposed desnowing and deraining approaches achieve significant improvements in comparison to the state-of-the-art approaches when tested on both synthetic and realistic datasets.