Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks

TL;DR

Weather GAN introduces a multi-domain weather translation method using GANs, enabling realistic transfer of weather conditions like sunny, rainy, or snowy in images by focusing on weather cues and maintaining semantic integrity.

Contribution

The paper proposes a novel multi-domain weather translation approach with a specialized generator that integrates attention and weather-cue segmentation modules for improved accuracy.

Findings

Outperforms existing methods in weather translation quality

Effectively focuses on weather cues to preserve image structure

Achieves realistic weather condition transfer across multiple categories

Abstract

In this paper, a new task is proposed, namely, weather translation, which refers to transferring weather conditions of the image from one category to another. It is important for photographic style transfer. Although lots of approaches have been proposed in traditional image translation tasks, few of them can handle the multi-category weather translation task, since weather conditions have rich categories and highly complex semantic structures. To address this problem, we develop a multi-domain weather translation approach based on generative adversarial networks (GAN), denoted as Weather GAN, which can achieve the transferring of weather conditions among sunny, cloudy, foggy, rainy and snowy. Specifically, the weather conditions in the image are determined by various weather-cues, such as cloud, blue sky, wet ground, etc. Therefore, it is essential for weather translation to focus the main attention on weather-cues. To this end, the generator of Weather GAN is composed of an initial translation module, an attention module and a weather-cue segmentation module. The initial translation module performs global translation during generation procedure. The weather-cue segmentation module identifies the structure and exact distribution of weather-cues. The attention module learns to focus on the interesting areas of the image while keeping other areas unaltered. The final generated result is synthesized by these three parts. This approach suppresses the distortion and deformation caused by weather translation. our approach outperforms the state-of-the-arts has been shown by a large number of experiments and evaluations.