SD-GAN: Structural and Denoising GAN reveals facial parts under occlusion

TL;DR

SD-GAN is a novel generative model that reconstructs occluded facial parts while preserving identity and illumination, significantly improving face recognition accuracy under occlusion.

Contribution

The paper introduces SD-GAN, a new GAN architecture with a structural loss and a novel training algorithm for effective face reconstruction under occlusion.

Findings

Outperforms existing methods on occluded face datasets

Enhances face recognition accuracy with reconstructed faces

Proves effective on both real and synthetic occlusions

Abstract

Certain facial parts are salient (unique) in appearance, which substantially contribute to the holistic recognition of a subject. Occlusion of these salient parts deteriorates the performance of face recognition algorithms. In this paper, we propose a generative model to reconstruct the missing parts of the face which are under occlusion. The proposed generative model (SD-GAN) reconstructs a face preserving the illumination variation and identity of the face. A novel adversarial training algorithm has been designed for a bimodal mutually exclusive Generative Adversarial Network (GAN) model, for faster convergence. A novel adversarial "structural" loss function is also proposed, comprising of two components: a holistic and a local loss, characterized by SSIM and patch-wise MSE. Ablation studies on real and synthetically occluded face datasets reveal that our proposed technique outperforms the competing methods by a considerable margin, even for boosting the performance of Face Recognition.