Boosting Facial Action Unit Detection Through Jointly Learning Facial Landmark Detection and Domain Separation and Reconstruction

TL;DR

This paper introduces a multi-task learning framework that jointly improves facial action unit detection by integrating facial landmark detection, domain separation, and reconstruction, leveraging contrastive learning for better feature alignment.

Contribution

It presents a novel multi-task learning approach with a new contrastive feature alignment scheme for enhanced AU detection in unconstrained settings.

Findings

Outperforms state-of-the-art AU detection methods on benchmark datasets.

Effectively leverages unlabeled data through domain separation and reconstruction.

Improves feature learning with contrastive supervision and intermediate constraints.

Abstract

Recently how to introduce large amounts of unlabeled facial images in the wild into supervised Facial Action Unit (AU) detection frameworks has become a challenging problem. In this paper, we propose a new AU detection framework where multi-task learning is introduced to jointly learn AU domain separation and reconstruction and facial landmark detection by sharing the parameters of homostructural facial extraction modules. In addition, we propose a new feature alignment scheme based on contrastive learning by simple projectors and an improved contrastive loss, which adds four additional intermediate supervisors to promote the feature reconstruction process. Experimental results on two benchmarks demonstrate our superiority against the state-of-the-art methods for AU detection in the wild.