Image Manipulation with Perceptual Discriminators

TL;DR

This paper introduces a perceptual discriminator architecture that combines perceptual losses with adversarial training for unaligned image translation, improving realism and robustness in image manipulation tasks.

Contribution

The paper presents a novel discriminator architecture embedding a pre-trained classification network, enabling effective unaligned image translation with combined perceptual and adversarial learning.

Findings

Outperforms baseline approaches in qualitative assessments.

Achieves superior quantitative metrics on unaligned image translation tasks.

Demonstrates robustness and efficiency of the combined architecture.

Abstract

Systems that perform image manipulation using deep convolutional networks have achieved remarkable realism. Perceptual losses and losses based on adversarial discriminators are the two main classes of learning objectives behind these advances. In this work, we show how these two ideas can be combined in a principled and non-additive manner for unaligned image translation tasks. This is accomplished through a special architecture of the discriminator network inside generative adversarial learning framework. The new architecture, that we call a perceptual discriminator, embeds the convolutional parts of a pre-trained deep classification network inside the discriminator network. The resulting architecture can be trained on unaligned image datasets while benefiting from the robustness and efficiency of perceptual losses. We demonstrate the merits of the new architecture in a series of qualitative and quantitative comparisons with baseline approaches and state-of-the-art frameworks for unaligned image translation.