Meta Auxiliary Learning for Facial Action Unit Detection

TL;DR

This paper introduces a Meta Auxiliary Learning method that enhances facial action unit detection by adaptively selecting related facial expression samples, reducing negative transfer and improving accuracy.

Contribution

The proposed MAL method automatically learns adaptive weights for auxiliary facial expression samples using meta learning, addressing negative transfer in multi-task AU detection.

Findings

MAL outperforms state-of-the-art multi-task methods on AU datasets.

Adaptive weighting reduces negative transfer from unrelated FE samples.

Experimental results confirm improved AU detection accuracy.

Abstract

Despite the success of deep neural networks on facial action unit (AU) detection, better performance depends on a large number of training images with accurate AU annotations. However, labeling AU is time-consuming, expensive, and error-prone. Considering AU detection and facial expression recognition (FER) are two highly correlated tasks, and facial expression (FE) is relatively easy to annotate, we consider learning AU detection and FER in a multi-task manner. However, the performance of the AU detection task cannot be always enhanced due to the negative transfer in the multi-task scenario. To alleviate this issue, we propose a Meta Auxiliary Learning method (MAL) that automatically selects highly related FE samples by learning adaptative weights for the training FE samples in a meta learning manner. The learned sample weights alleviate the negative transfer from two aspects: 1) balance the loss of each task automatically, and 2) suppress the weights of FE samples that have large uncertainties. Experimental results on several popular AU datasets demonstrate MAL consistently improves the AU detection performance compared with the state-of-the-art multi-task and auxiliary learning methods. MAL automatically estimates adaptive weights for the auxiliary FE samples according to their semantic relevance with the primary AU detection task.