Norface: Improving Facial Expression Analysis by Identity Normalization

TL;DR

Norface introduces a unified framework that normalizes facial identity and pose to improve the accuracy of facial expression analysis tasks such as AU detection and FER, outperforming state-of-the-art methods.

Contribution

The paper presents a novel identity normalization framework with a normalization network and a classification network using Mixture of Experts, enhancing expression analysis accuracy.

Findings

Outperforms existing SOTA methods in AU detection and FER.

Effectively normalizes identity, pose, and background for better analysis.

Improves cross-dataset facial expression recognition performance.

Abstract

Facial Expression Analysis remains a challenging task due to unexpected task-irrelevant noise, such as identity, head pose, and background. To address this issue, this paper proposes a novel framework, called Norface, that is unified for both Action Unit (AU) analysis and Facial Emotion Recognition (FER) tasks. Norface consists of a normalization network and a classification network. First, the carefully designed normalization network struggles to directly remove the above task-irrelevant noise, by maintaining facial expression consistency but normalizing all original images to a common identity with consistent pose, and background. Then, these additional normalized images are fed into the classification network. Due to consistent identity and other factors (e.g. head pose, background, etc.), the normalized images enable the classification network to extract useful expression information more effectively. Additionally, the classification network incorporates a Mixture of Experts to refine the latent representation, including handling the input of facial representations and the output of multiple (AU or emotion) labels. Extensive experiments validate the carefully designed framework with the insight of identity normalization. The proposed method outperforms existing SOTA methods in multiple facial expression analysis tasks, including AU detection, AU intensity estimation, and FER tasks, as well as their cross-dataset tasks. For the normalized datasets and code please visit {https://norface-fea.github.io/}.