Micro-Expression Recognition Based on Attribute Information Embedding and Cross-modal Contrastive Learning

TL;DR

This paper introduces a novel micro-expression recognition approach combining attribute information embedding and cross-modal contrastive learning, leveraging multi-modal features to improve accuracy with limited data.

Contribution

The paper proposes a new method that integrates attribute embedding and cross-modal contrastive learning for enhanced micro-expression recognition.

Findings

Achieved 77.82% accuracy on CASME II database.

Achieved 71.04% accuracy on MMEW database.

Outperforms existing methods in micro-expression recognition.

Abstract

Facial micro-expressions recognition has attracted much attention recently. Micro-expressions have the characteristics of short duration and low intensity, and it is difficult to train a high-performance classifier with the limited number of existing micro-expressions. Therefore, recognizing micro-expressions is a challenge task. In this paper, we propose a micro-expression recognition method based on attribute information embedding and cross-modal contrastive learning. We use 3D CNN to extract RGB features and FLOW features of micro-expression sequences and fuse them, and use BERT network to extract text information in Facial Action Coding System. Through cross-modal contrastive loss, we embed attribute information in the visual network, thereby improving the representation ability of micro-expression recognition in the case of limited samples. We conduct extensive experiments in CASME II and MMEW databases, and the accuracy is 77.82% and 71.04%, respectively. The comparative experiments show that this method has better recognition effect than other methods for micro-expression recognition.