# Perceptual Adversarial Networks for Image-to-Image Transformation

## TL;DR

This paper introduces Perceptual Adversarial Networks (PAN), a versatile framework for image-to-image transformation tasks that combines generative and perceptual adversarial losses to improve transformation quality.

## Contribution

The paper presents a generic adversarial learning framework with a perceptual loss for various image-to-image transformations, outperforming existing methods.

## Key findings

- PAN outperforms state-of-the-art methods in multiple tasks
- The framework effectively narrows the gap between transformed and ground-truth images
- Adversarial training with perceptual loss enhances image quality

## Abstract

In this paper, we propose a principled Perceptual Adversarial Networks (PAN) for image-to-image transformation tasks. Unlike existing application-specific algorithms, PAN provides a generic framework of learning mapping relationship between paired images (Fig. 1), such as mapping a rainy image to its de-rained counterpart, object edges to its photo, semantic labels to a scenes image, etc. The proposed PAN consists of two feed-forward convolutional neural networks (CNNs), the image transformation network T and the discriminative network D. Through combining the generative adversarial loss and the proposed perceptual adversarial loss, these two networks can be trained alternately to solve image-to-image transformation tasks. Among them, the hidden layers and output of the discriminative network D are upgraded to continually and automatically discover the discrepancy between the transformed image and the corresponding ground-truth. Simultaneously, the image transformation network T is trained to minimize the discrepancy explored by the discriminative network D. Through the adversarial training process, the image transformation network T will continually narrow the gap between transformed images and ground-truth images. Experiments evaluated on several image-to-image transformation tasks (e.g., image de-raining, image inpainting, etc.) show that the proposed PAN outperforms many related state-of-the-art methods.