Micro-Expression Recognition by Motion Feature Extraction based on Pre-training

TL;DR

This paper introduces a novel motion feature extraction strategy using pre-training to improve micro-expression recognition, addressing challenges like subtle motions and limited data, and demonstrates superior performance on multiple datasets.

Contribution

The paper proposes MoExt, a pre-training based motion extraction method that enhances feature separation and recognition accuracy in micro-expression analysis.

Findings

Outperforms state-of-the-art methods on CASME II, SMIC, SAMM, and CAS(ME)3 datasets.

Effectively extracts inter-frame motion features while excluding irrelevant information.

Improves recognition accuracy in challenging micro-expression scenarios.

Abstract

Micro-expressions (MEs) are spontaneous, unconscious facial expressions that have promising applications in various fields such as psychotherapy and national security. Thus, micro-expression recognition (MER) has attracted more and more attention from researchers. Although various MER methods have emerged especially with the development of deep learning techniques, the task still faces several challenges, e.g. subtle motion and limited training data. To address these problems, we propose a novel motion extraction strategy (MoExt) for the MER task and use additional macro-expression data in the pre-training process. We primarily pretrain the feature separator and motion extractor using the contrastive loss, thus enabling them to extract representative motion features. In MoExt, shape features and texture features are first extracted separately from onset and apex frames, and then motion features related to MEs are extracted based on the shape features of both frames. To enable the model to more effectively separate features, we utilize the extracted motion features and the texture features from the onset frame to reconstruct the apex frame. Through pre-training, the module is enabled to extract inter-frame motion features of facial expressions while excluding irrelevant information. The feature separator and motion extractor are ultimately integrated into the MER network, which is then fine-tuned using the target ME data. The effectiveness of proposed method is validated on three commonly used datasets, i.e., CASME II, SMIC, SAMM, and CAS(ME)3 dataset. The results show that our method performs favorably against state-of-the-art methods.