Probabilistic Attribute Tree in Convolutional Neural Networks for Facial Expression Recognition

TL;DR

This paper introduces a Probabilistic Attribute Tree-CNN that explicitly models identity-related attributes to improve facial expression recognition, especially in unconstrained environments, achieving state-of-the-art results.

Contribution

The paper presents a novel hierarchical PAT module with a probabilistic learning strategy and semi-supervised training, enhancing expression recognition by handling attribute variations.

Findings

Outperforms baseline models on five datasets

Achieves top accuracy on SFEW in-the-wild dataset

Effective semi-supervised learning from limited attribute data

Abstract

In this paper, we proposed a novel Probabilistic Attribute Tree-CNN (PAT-CNN) to explicitly deal with the large intra-class variations caused by identity-related attributes, e.g., age, race, and gender. Specifically, a novel PAT module with an associated PAT loss was proposed to learn features in a hierarchical tree structure organized according to attributes, where the final features are less affected by the attributes. Then, expression-related features are extracted from leaf nodes. Samples are probabilistically assigned to tree nodes at different levels such that expression-related features can be learned from all samples weighted by probabilities. We further proposed a semi-supervised strategy to learn the PAT-CNN from limited attribute-annotated samples to make the best use of available data. Experimental results on five facial expression datasets have demonstrated that the proposed PAT-CNN outperforms the baseline models by explicitly modeling attributes. More impressively, the PAT-CNN using a single model achieves the best performance for faces in the wild on the SFEW dataset, compared with the state-of-the-art methods using an ensemble of hundreds of CNNs.