Taming Adversarial Domain Transfer with Structural Constraints for Image Enhancement

TL;DR

This paper proposes methods to improve unsupervised domain transfer for image enhancement in traffic scenes, addressing structural distortions caused by adversarial training, to better restore degraded images without pixel-perfect pairs.

Contribution

It introduces three novel approaches to preserve scene structure during adversarial domain transfer for image enhancement under various degradations.

Findings

Enhanced image quality in degraded traffic scenes

Reduced structural distortions in generated images

Improved semantic consistency in domain transfer

Abstract

The goal of this work is to improve images of traffic scenes that are degraded by natural causes such as fog, rain and limited visibility during the night. For these applications, it is next to impossible to get pixel perfect pairs of the same scene, with and without the degrading conditions. This makes it unsuitable for conventional supervised learning approaches, however, it is easy to collect unpaired images of the scenes in a perfect and in a degraded condition. To enhance the images taken in a poor visibility condition, domain transfer models can be trained to transform an image from the degraded to the clear domain. A well-known concept for unsupervised domain transfer are cycle-consistent generative adversarial models. Unfortunately, the resulting generators often change the structure of the scene. This causes an undesirable change in the semantics. We propose three ways to cope with this problem depending on the type of degradation.