# Multi-Task Convolutional Neural Network for Pose-Invariant Face Recognition

## TL;DR

This paper introduces a multi-task CNN for pose-invariant face recognition, utilizing dynamic weighting and pose-specific features, demonstrating improved performance on multiple datasets.

## Contribution

It proposes a novel multi-task CNN with dynamic loss weighting and pose grouping, advancing pose-invariant face recognition techniques.

## Key findings

- Effective disentanglement of identity and pose variations.
- Superior performance on Multi-PIE, LFW, CFP, and IJB-A datasets.
- First to use all Multi-PIE data for face recognition.

## Abstract

This paper explores multi-task learning (MTL) for face recognition. We answer the questions of how and why MTL can improve the face recognition performance. First, we propose a multi-task Convolutional Neural Network (CNN) for face recognition where identity classification is the main task and pose, illumination, and expression estimations are the side tasks. Second, we develop a dynamic-weighting scheme to automatically assign the loss weight to each side task, which is a crucial problem in MTL. Third, we propose a pose-directed multi-task CNN by grouping different poses to learn pose-specific identity features, simultaneously across all poses. Last but not least, we propose an energy-based weight analysis method to explore how CNN-based MTL works. We observe that the side tasks serve as regularizations to disentangle the variations from the learnt identity features. Extensive experiments on the entire Multi-PIE dataset demonstrate the effectiveness of the proposed approach. To the best of our knowledge, this is the first work using all data in Multi-PIE for face recognition. Our approach is also applicable to in-the-wild datasets for pose-invariant face recognition and achieves comparable or better performance than state of the art on LFW, CFP, and IJB-A datasets.