Cross-Dataset Facial Micro-Expression Recognition with Regularization Learning and Action Unit-Guided Data Augmentation

TL;DR

This paper introduces new methods to improve facial micro-expression recognition across different datasets by addressing feature distribution and data imbalance issues.

Contribution

The paper proposes a regularization learning module and an Action Unit-guided GAN for cross-dataset micro-expression recognition.

Findings

The regularization module helps learn domain-invariant representations and prevents overfitting.

The AU-guided GAN effectively mitigates data imbalance by generating balanced micro-expression samples.

The proposed methods outperform state-of-the-art approaches in cross-dataset recognition tasks.

Abstract

With the growing development of facial micro-expression recognition technology, its practical application value has attracted increasing attention. In real-world scenarios, facial micro-expression recognition typically involves cross-dataset evaluation, where training and testing samples come from different datasets. Specifically, cross-dataset micro-expression recognition employs multi-dataset composite training and unseen single-dataset testing. This setup introduces two major challenges: inconsistent feature distributions across training sets and data imbalance. To address the distribution discrepancy of the same category across different training datasets, we propose a plug-and-play batch regularization learning module that constrains weight discrepancies across datasets through information-theoretic regularization, facilitating the learning of domain-invariant representations while preventing overfitting to specific source domains. To mitigate the data imbalance issue, we propose an Action Unit (AU)-guided generative adversarial network (GAN) for synthesizing micro-expression samples. This approach uses K-means clustering to obtain cluster centers of AU intensities for each category, which are then used to guide the GAN in generating balanced micro-expression samples. To validate the effectiveness of the proposed methods, extensive experiments are conducted on CNN, ResNet, and PoolFormer architectures. The results demonstrate that our approach achieves superior performance in cross-dataset recognition compared to state-of-the-art methods.

Figures