FED-PsyAU: Privacy-Preserving Micro-Expression Recognition via Psychological AU Coordination and Dynamic Facial Motion Modeling

TL;DR

This paper introduces FED-PsyAU, a privacy-preserving micro-expression recognition framework that leverages psychological AU coordination and dynamic facial motion modeling, improving recognition accuracy while maintaining data privacy.

Contribution

It proposes a novel federated learning approach combined with psychological AU priors and dynamic facial motion modeling for micro-expression recognition.

Findings

Effective micro-expression recognition across multiple clients.

Enhanced recognition accuracy with privacy preservation.

Robustness to limited sample sizes.

Abstract

Micro-expressions (MEs) are brief, low-intensity, often localized facial expressions. They could reveal genuine emotions individuals may attempt to conceal, valuable in contexts like criminal interrogation and psychological counseling. However, ME recognition (MER) faces challenges, such as small sample sizes and subtle features, which hinder efficient modeling. Additionally, real-world applications encounter ME data privacy issues, leaving the task of enhancing recognition across settings under privacy constraints largely unexplored. To address these issues, we propose a FED-PsyAU research framework. We begin with a psychological study on the coordination of upper and lower facial action units (AUs) to provide structured prior knowledge of facial muscle dynamics. We then develop a DPK-GAT network that combines these psychological priors with statistical AU patterns, enabling hierarchical learning of facial motion features from regional to global levels, effectively enhancing MER performance. Additionally, our federated learning framework advances MER capabilities across multiple clients without data sharing, preserving privacy and alleviating the limited-sample issue for each client. Extensive experiments on commonly-used ME databases demonstrate the effectiveness of our approach.