Global-to-local Expression-aware Embeddings for Facial Action Unit Detection

TL;DR

This paper introduces a novel global-to-local embedding approach that captures subtle facial movements for improved facial action unit detection, outperforming previous methods on standard datasets.

Contribution

The paper proposes a global expression encoder trained on large-scale data and a local AU feature module to enhance AU detection accuracy.

Findings

Achieves state-of-the-art performance on BP4D, DISFA, and BP4D+ datasets.

Outperforms previous methods in AU detection accuracy.

Effectively captures subtle expression changes through global and local features.

Abstract

Expressions and facial action units (AUs) are two levels of facial behavior descriptors. Expression auxiliary information has been widely used to improve the AU detection performance. However, most existing expression representations can only describe pre-determined discrete categories (e.g., Angry, Disgust, Happy, Sad, etc.) and cannot capture subtle expression transformations like AUs. In this paper, we propose a novel fine-grained \textsl{Global Expression representation Encoder} to capture subtle and continuous facial movements, to promote AU detection. To obtain such a global expression representation, we propose to train an expression embedding model on a large-scale expression dataset according to global expression similarity. Moreover, considering the local definition of AUs, it is essential to extract local AU features. Therefore, we design a \textsl{Local AU Features Module} to generate local facial features for each AU. Specifically, it consists of an AU feature map extractor and a corresponding AU mask extractor. First, the two extractors transform the global expression representation into AU feature maps and masks, respectively. Then, AU feature maps and their corresponding AU masks are multiplied to generate AU masked features focusing on local facial region. Finally, the AU masked features are fed into an AU classifier for judging the AU occurrence. Extensive experiment results demonstrate the superiority of our proposed method. Our method validly outperforms previous works and achieves state-of-the-art performances on widely-used face datasets, including BP4D, DISFA, and BP4D+.