MEDN: Motion-Emotion Feature Decoupling Network for Micro-Expression Recognition

TL;DR

This paper introduces MEDN, a novel dual-branch network that decouples motion and emotion features for improved micro-expression recognition, leveraging explicit AU detection and a sparse vision transformer.

Contribution

The paper proposes a new framework with orthogonal loss and a sparse transformer to better separate motion and emotion features in micro-expression recognition.

Findings

MEDN outperforms existing methods on three benchmark datasets.

Decoupling motion and emotion features improves recognition accuracy.

The approach enhances generalization in micro-expression recognition.

Abstract

Unlike macro-expression, micro-expression does not follow a strictly consistent mapping rule between emotions and Action Units (AUs). As a result, some micro-expressions share identical AUs yet represent completely opposite emotional categories, making them highly visually similar. Existing microexpression recognition (MER) methods mostly rely on explicit facial motion cues (e.g., optical flow, frame differences, AU features) while ignoring implicit emotion information. To tackle this issue, this paper presents a Motion Emotion Feature Decoupling Network (MEDN) for MER. We design a dual-branch framework to separately extract motion and emotion features. In the motion branch, an AU-detection task restricts features to the explicit motion domain, and orthogonal loss is adopted to reduce motion emotion feature coupling. For implicit emotion modeling, we propose a Sparse Emotion Vision Transformer (SEVit) that sparsifies spatial tokens to highlight local temporal variations with multi-scale sparsity rates. A Collaborative Fusion Module (CoFM) is further developed to fuse disentangled motion and emotion features adaptively. Extensive experiments on three benchmark datasets validate that MEDN effectively decouples motion and emotion features and achieves superior recognition performance, offering a new perspective for enhancing recognition accuracy and generalization.