Pose-Robust Face Recognition via Deep Residual Equivariant Mapping

TL;DR

This paper introduces DREAM, a residual equivariant mapping block that improves profile face recognition by bridging pose discrepancies in deep representations without extensive data augmentation.

Contribution

The paper proposes a novel DREAM block that adaptively transforms profile face representations to canonical poses, enhancing recognition accuracy across various deep networks.

Findings

DREAM improves profile face recognition performance.

The method is lightweight and adds negligible computational overhead.

It works effectively with multiple deep network architectures.

Abstract

Face recognition achieves exceptional success thanks to the emergence of deep learning. However, many contemporary face recognition models still perform relatively poor in processing profile faces compared to frontal faces. A key reason is that the number of frontal and profile training faces are highly imbalanced - there are extensively more frontal training samples compared to profile ones. In addition, it is intrinsically hard to learn a deep representation that is geometrically invariant to large pose variations. In this study, we hypothesize that there is an inherent mapping between frontal and profile faces, and consequently, their discrepancy in the deep representation space can be bridged by an equivariant mapping. To exploit this mapping, we formulate a novel Deep Residual EquivAriant Mapping (DREAM) block, which is capable of adaptively adding residuals to the input deep representation to transform a profile face representation to a canonical pose that simplifies recognition. The DREAM block consistently enhances the performance of profile face recognition for many strong deep networks, including ResNet models, without deliberately augmenting training data of profile faces. The block is easy to use, light-weight, and can be implemented with a negligible computational overhead.