Teacher-Student Adversarial Depth Hallucination to Improve Face Recognition

TL;DR

This paper introduces TS-GAN, a teacher-student adversarial framework that generates depth images from RGB inputs to enhance face recognition accuracy across multiple datasets.

Contribution

The novel TS-GAN architecture enables effective depth hallucination from RGB images, improving face recognition performance and generalization to unseen datasets.

Findings

TS-GAN outperforms existing methods in synthetic depth image generation.

Hallucinated depth images improve face recognition accuracy by up to 2.6%.

The shared generator in TS-GAN is effective for real-time depth hallucination.

Abstract

We present the Teacher-Student Generative Adversarial Network (TS-GAN) to generate depth images from single RGB images in order to boost the performance of face recognition systems. For our method to generalize well across unseen datasets, we design two components in the architecture, a teacher and a student. The teacher, which itself consists of a generator and a discriminator, learns a latent mapping between input RGB and paired depth images in a supervised fashion. The student, which consists of two generators (one shared with the teacher) and a discriminator, learns from new RGB data with no available paired depth information, for improved generalization. The fully trained shared generator can then be used in runtime to hallucinate depth from RGB for downstream applications such as face recognition. We perform rigorous experiments to show the superiority of TS-GAN over other methods in generating synthetic depth images. Moreover, face recognition experiments demonstrate that our hallucinated depth along with the input RGB images boosts performance across various architectures when compared to a single RGB modality by average values of +1.2%, +2.6%, and +2.6% for IIIT-D, EURECOM, and LFW datasets respectively. We make our implementation public at: https://github.com/hardik-uppal/teacher-student-gan.git.