Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN

TL;DR

This paper introduces AU-GAN, an asymmetric GAN model with uncertainty-aware cycle-consistency loss for improved adverse weather image translation, addressing limitations of symmetric architectures in handling imbalanced domain information.

Contribution

The paper proposes a novel asymmetric GAN architecture with feature transfer and uncertainty modeling for better adverse weather image translation.

Findings

AU-GAN outperforms state-of-the-art models in qualitative assessments.

AU-GAN achieves superior quantitative translation metrics.

The asymmetric design effectively handles imbalanced domain information.

Abstract

Adverse weather image translation belongs to the unsupervised image-to-image (I2I) translation task which aims to transfer adverse condition domain (eg, rainy night) to standard domain (eg, day). It is a challenging task because images from adverse domains have some artifacts and insufficient information. Recently, many studies employing Generative Adversarial Networks (GANs) have achieved notable success in I2I translation but there are still limitations in applying them to adverse weather enhancement. Symmetric architecture based on bidirectional cycle-consistency loss is adopted as a standard framework for unsupervised domain transfer methods. However, it can lead to inferior translation result if the two domains have imbalanced information. To address this issue, we propose a novel GAN model, i.e., AU-GAN, which has an asymmetric architecture for adverse domain translation. We insert a proposed feature transfer network (${T}$-net) in only a normal domain generator (i.e., rainy night-> day) to enhance encoded features of the adverse domain image. In addition, we introduce asymmetric feature matching for disentanglement of encoded features. Finally, we propose uncertainty-aware cycle-consistency loss to address the regional uncertainty of a cyclic reconstructed image. We demonstrate the effectiveness of our method by qualitative and quantitative comparisons with state-of-the-art models. Codes are available at https://github.com/jgkwak95/AU-GAN.