Load Balanced GANs for Multi-view Face Image Synthesis

TL;DR

This paper introduces LB-GAN, a novel approach for multi-view face synthesis that decomposes the task into normalization and editing stages, achieving realistic, identity-preserving images at arbitrary angles.

Contribution

The paper proposes Load Balanced GANs that decompose face rotation into normalization and editing, improving realism and identity preservation in multi-view face synthesis.

Findings

Enhanced visual realism of synthesized images

Better preservation of identity information

Effective rotation to arbitrary angles

Abstract

Multi-view face synthesis from a single image is an ill-posed problem and often suffers from serious appearance distortion. Producing photo-realistic and identity preserving multi-view results is still a not well defined synthesis problem. This paper proposes Load Balanced Generative Adversarial Networks (LB-GAN) to precisely rotate the yaw angle of an input face image to any specified angle. LB-GAN decomposes the challenging synthesis problem into two well constrained subtasks that correspond to a face normalizer and a face editor respectively. The normalizer first frontalizes an input image, and then the editor rotates the frontalized image to a desired pose guided by a remote code. In order to generate photo-realistic local details, the normalizer and the editor are trained in a two-stage manner and regulated by a conditional self-cycle loss and an attention based L2 loss. Exhaustive experiments on controlled and uncontrolled environments demonstrate that the proposed method not only improves the visual realism of multi-view synthetic images, but also preserves identity information well.