# Unconstrained Facial Action Unit Detection via Latent Feature Domain

## TL;DR

This paper introduces an end-to-end domain adaptation framework for unconstrained facial action unit detection that transfers labels from a constrained source to an unconstrained target domain using latent feature mapping and adversarial learning.

## Contribution

It proposes a novel latent feature domain approach with landmark-based label transfer and a landmark adversarial loss for disentangling features, advancing in-the-wild AU detection.

## Key findings

- Outperforms state-of-the-art methods on in-the-wild benchmarks.
- Effectively transfers AU labels across domains with improved accuracy.
- Demonstrates robustness to unconstrained facial appearance variability.

## Abstract

Facial action unit (AU) detection in the wild is a challenging problem, due to the unconstrained variability in facial appearances and the lack of accurate annotations. Most existing methods depend on either impractical labor-intensive labeling or inaccurate pseudo labels. In this paper, we propose an end-to-end unconstrained facial AU detection framework based on domain adaptation, which transfers accurate AU labels from a constrained source domain to an unconstrained target domain by exploiting labels of AU-related facial landmarks. Specifically, we map a source image with label and a target image without label into a latent feature domain by combining source landmark-related feature with target landmark-free feature. Due to the combination of source AU-related information and target AU-free information, the latent feature domain with transferred source label can be learned by maximizing the target-domain AU detection performance. Moreover, we introduce a novel landmark adversarial loss to disentangle the landmark-free feature from the landmark-related feature by treating the adversarial learning as a multi-player minimax game. Our framework can also be naturally extended for use with target-domain pseudo AU labels. Extensive experiments show that our method soundly outperforms lower-bounds and upper-bounds of the basic model, as well as state-of-the-art approaches on the challenging in-the-wild benchmarks. The code is available at https://github.com/ZhiwenShao/ADLD.