Pairwise-GAN: Pose-based View Synthesis through Pair-Wise Training

TL;DR

This paper introduces Pairwise-GAN, a novel model for pose-based face frontalization that outperforms existing methods like CycleGAN and Pix2Pix in face similarity metrics, advancing face synthesis in unconstrained scenarios.

Contribution

The paper proposes Pairwise-GAN, a new dual-U-Net generator architecture with PatchGAN discriminator, improving face frontalization performance over prior pixel transformation models.

Findings

Pairwise-GAN outperforms CycleGAN by 5.4% in face similarity.

Pix2Pix with combined loss functions improves by 2.72%.

The proposed method enhances face synthesis quality in the wild.

Abstract

Three-dimensional face reconstruction is one of the popular applications in computer vision. However, even state-of-the-art models still require frontal face as inputs, which restricts its usage scenarios in the wild. A similar dilemma also happens in face recognition. New research designed to recover the frontal face from a single side-pose facial image has emerged. The state-of-the-art in this area is the Face-Transformation generative adversarial network, which is based on the CycleGAN. This inspired our research which explores the performance of two models from pixel transformation in frontal facial synthesis, Pix2Pix and CycleGAN. We conducted the experiments on five different loss functions on Pix2Pix to improve its performance, then followed by proposing a new network Pairwise-GAN in frontal facial synthesis. Pairwise-GAN uses two parallel U-Nets as the generator and PatchGAN as the discriminator. The detailed hyper-parameters are also discussed. Based on the quantitative measurement by face similarity comparison, our results showed that Pix2Pix with L1 loss, gradient difference loss, and identity loss results in 2.72% of improvement at average similarity compared to the default Pix2Pix model. Additionally, the performance of Pairwise-GAN is 5.4% better than the CycleGAN and 9.1% than the Pix2Pix at average similarity.