FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features

TL;DR

FG-Net leverages pre-trained StyleGAN2 features and a Pyramid CNN to improve generalization in facial Action Unit detection across different datasets, even with limited training data.

Contribution

The paper introduces FG-Net, a novel AU detection framework that uses features from a pre-trained generative model for enhanced cross-domain generalization.

Findings

Superior cross-corpus AU detection performance

Effective with limited training samples (as few as 1000)

Maintains competitive within-domain accuracy

Abstract

Automatic detection of facial Action Units (AUs) allows for objective facial expression analysis. Due to the high cost of AU labeling and the limited size of existing benchmarks, previous AU detection methods tend to overfit the dataset, resulting in a significant performance loss when evaluated across corpora. To address this problem, we propose FG-Net for generalizable facial action unit detection. Specifically, FG-Net extracts feature maps from a StyleGAN2 model pre-trained on a large and diverse face image dataset. Then, these features are used to detect AUs with a Pyramid CNN Interpreter, making the training efficient and capturing essential local features. The proposed FG-Net achieves a strong generalization ability for heatmap-based AU detection thanks to the generalizable and semantic-rich features extracted from the pre-trained generative model. Extensive experiments are conducted to evaluate within- and cross-corpus AU detection with the widely-used DISFA and BP4D datasets. Compared with the state-of-the-art, the proposed method achieves superior cross-domain performance while maintaining competitive within-domain performance. In addition, FG-Net is data-efficient and achieves competitive performance even when trained on 1000 samples. Our code will be released at \url{https://github.com/ihp-lab/FG-Net}