ARPGNet: Appearance- and Relation-aware Parallel Graph Attention Fusion Network for Facial Expression Recognition

TL;DR

ARPGNet introduces a novel graph attention fusion network that effectively models facial region relationships and appearance dynamics to improve facial expression recognition accuracy.

Contribution

This paper proposes a new parallel graph attention fusion network that jointly learns appearance and relation features for facial expression recognition.

Findings

ARPGNet outperforms existing methods on three datasets.

The relational graph modeling enhances expression recognition accuracy.

Parallel fusion of appearance and relation features improves robustness.

Abstract

The key to facial expression recognition is to learn discriminative spatial-temporal representations that embed facial expression dynamics. Previous studies predominantly rely on pre-trained Convolutional Neural Networks (CNNs) to learn facial appearance representations, overlooking the relationships between facial regions. To address this issue, this paper presents an Appearance- and Relation-aware Parallel Graph attention fusion Network (ARPGNet) to learn mutually enhanced spatial-temporal representations of appearance and relation information. Specifically, we construct a facial region relation graph and leverage the graph attention mechanism to model the relationships between facial regions. The resulting relational representation sequences, along with CNN-based appearance representation sequences, are then fed into a parallel graph attention fusion module for mutual interaction and enhancement. This module simultaneously explores the complementarity between different representation sequences and the temporal dynamics within each sequence. Experimental results on three facial expression recognition datasets demonstrate that the proposed ARPGNet outperforms or is comparable to state-of-the-art methods.