Disentangling Identity and Pose for Facial Expression Recognition

TL;DR

This paper introduces a novel method for facial expression recognition that effectively disentangles identity, pose, and expression features, leading to improved accuracy on diverse datasets by leveraging pre-trained face recognition models and a reconstruction-based disentanglement approach.

Contribution

The work presents a new IPD-FER model that disentangles identity and pose from expression, enabling robust FER on in-the-wild data without restrictive training data.

Findings

Achieves state-of-the-art FER accuracy on multiple datasets.

Effectively disentangles identity, pose, and expression features.

Demonstrates robustness in real-world, in-the-wild scenarios.

Abstract

Facial expression recognition (FER) is a challenging problem because the expression component is always entangled with other irrelevant factors, such as identity and head pose. In this work, we propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. We regard the holistic facial representation as the combination of identity, pose and expression. These three components are encoded with different encoders. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data in previous works and makes the disentanglement practicable on in-the-wild datasets. At the same time, the pose and expression encoder are optimized with corresponding labels. Combining identity and pose feature, a neutral face of input individual should be generated by the decoder. When expression feature is added, the input image should be reconstructed. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose. Experimental results verify the effectiveness of our method on both lab-controlled and in-the-wild databases and we achieve state-of-the-art recognition performance.