Facial Action Unit Detection via Adaptive Attention and Relation

TL;DR

This paper introduces an adaptive attention and relation framework for facial action unit detection, effectively capturing both local and global dependencies, and reasoning about spatial-temporal AU relationships to improve detection accuracy.

Contribution

The paper proposes a novel adaptive attention regression network and a spatio-temporal graph convolutional network tailored for AU detection, addressing limitations of prior methods.

Findings

Achieves competitive performance on BP4D, DISFA, GFT, and Aff-Wild2 datasets.

Precisely learns regional correlation distributions of each AU.

Effectively models spatial and temporal dependencies in AU detection.

Abstract

Facial action unit (AU) detection is challenging due to the difficulty in capturing correlated information from subtle and dynamic AUs. Existing methods often resort to the localization of correlated regions of AUs, in which predefining local AU attentions by correlated facial landmarks often discards essential parts, or learning global attention maps often contains irrelevant areas. Furthermore, existing relational reasoning methods often employ common patterns for all AUs while ignoring the specific way of each AU. To tackle these limitations, we propose a novel adaptive attention and relation (AAR) framework for facial AU detection. Specifically, we propose an adaptive attention regression network to regress the global attention map of each AU under the constraint of attention predefinition and the guidance of AU detection, which is beneficial for capturing both specified dependencies by landmarks in strongly correlated regions and facial globally distributed dependencies in weakly correlated regions. Moreover, considering the diversity and dynamics of AUs, we propose an adaptive spatio-temporal graph convolutional network to simultaneously reason the independent pattern of each AU, the inter-dependencies among AUs, as well as the temporal dependencies. Extensive experiments show that our approach (i) achieves competitive performance on challenging benchmarks including BP4D, DISFA, and GFT in constrained scenarios and Aff-Wild2 in unconstrained scenarios, and (ii) can precisely learn the regional correlation distribution of each AU.