Deep Learning Multi-View Representation for Face Recognition

TL;DR

This paper introduces a deep neural network called multi-view perceptron (MVP) that disentangles identity and view features in face images, enabling recognition and view synthesis from a single 2D image, inspired by human brain capabilities.

Contribution

The paper presents a novel deep learning model that separates identity and view representations and can generate multi-view face images without 3D data, mimicking human perception.

Findings

Achieves superior identity recognition on MultiPIE dataset.

Can interpolate and predict face images under unseen viewpoints.

Demonstrates robustness to view changes in face recognition.

Abstract

Various factors, such as identities, views (poses), and illuminations, are coupled in face images. Disentangling the identity and view representations is a major challenge in face recognition. Existing face recognition systems either use handcrafted features or learn features discriminatively to improve recognition accuracy. This is different from the behavior of human brain. Intriguingly, even without accessing 3D data, human not only can recognize face identity, but can also imagine face images of a person under different viewpoints given a single 2D image, making face perception in the brain robust to view changes. In this sense, human brain has learned and encoded 3D face models from 2D images. To take into account this instinct, this paper proposes a novel deep neural net, named multi-view perceptron (MVP), which can untangle the identity and view features, and infer a full spectrum of multi-view images in the meanwhile, given a single 2D face image. The identity features of MVP achieve superior performance on the MultiPIE dataset. MVP is also capable to interpolate and predict images under viewpoints that are unobserved in the training data.