Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection

TL;DR

This paper introduces an end-to-end deep learning framework utilizing graph convolutional networks to model relationships among facial action units, significantly improving detection accuracy over previous methods.

Contribution

It is the first to integrate GCNs with deep learning for AU relation modeling in an end-to-end manner, replacing traditional probabilistic graphical models.

Findings

Outperforms state-of-the-art methods on BP4D and DISFA datasets.

Demonstrates the effectiveness of GCN-based AU relation modeling.

Validated through comprehensive ablation studies.

Abstract

Most existing AU detection works considering AU relationships are relying on probabilistic graphical models with manually extracted features. This paper proposes an end-to-end deep learning framework for facial AU detection with graph convolutional network (GCN) for AU relation modeling, which has not been explored before. In particular, AU related regions are extracted firstly, latent representations full of AU information are learned through an auto-encoder. Moreover, each latent representation vector is feed into GCN as a node, the connection mode of GCN is determined based on the relationships of AUs. Finally, the assembled features updated through GCN are concatenated for AU detection. Extensive experiments on BP4D and DISFA benchmarks demonstrate that our framework significantly outperforms the state-of-the-art methods for facial AU detection. The proposed framework is also validated through a series of ablation studies.