Reconstruction-Based Disentanglement for Pose-invariant Face Recognition

TL;DR

This paper introduces a pose-invariant face recognition method that generates non-frontal views, learns disentangled features, and explicitly separates identity from pose, improving recognition accuracy across large pose variations.

Contribution

It proposes a novel reconstruction-based disentanglement approach that does not require extensive pose coverage in training data.

Findings

Outperforms state-of-the-art on multiple face datasets

Effective in handling large pose variations

Improves recognition accuracy with pose-invariant features

Abstract

Deep neural networks (DNNs) trained on large-scale datasets have recently achieved impressive improvements in face recognition. But a persistent challenge remains to develop methods capable of handling large pose variations that are relatively underrepresented in training data. This paper presents a method for learning a feature representation that is invariant to pose, without requiring extensive pose coverage in training data. We first propose to generate non-frontal views from a single frontal face, in order to increase the diversity of training data while preserving accurate facial details that are critical for identity discrimination. Our next contribution is to seek a rich embedding that encodes identity features, as well as non-identity ones such as pose and landmark locations. Finally, we propose a new feature reconstruction metric learning to explicitly disentangle identity and pose, by demanding alignment between the feature reconstructions through various combinations of identity and pose features, which is obtained from two images of the same subject. Experiments on both controlled and in-the-wild face datasets, such as MultiPIE, 300WLP and the profile view database CFP, show that our method consistently outperforms the state-of-the-art, especially on images with large head pose variations. Detail results and resource are referred to https://sites.google.com/site/xipengcshomepage/iccv2017