Evaluation of the Spatio-Temporal features and GAN for Micro-expression Recognition System

TL;DR

This paper reviews and enhances micro-expression recognition using optical flow features and GAN-generated data, proposing a modified CNN and evaluating on multiple datasets to improve accuracy.

Contribution

It introduces a novel combination of optical flow variations, GAN-based data augmentation, and a modified CNN for improved micro-expression recognition.

Findings

GAN data augmentation improves recognition accuracy.

Modified CNN outperforms baseline models.

Results validated on SMIC, CASME II, and SAMM datasets.

Abstract

Owing to the development and advancement of artificial intelligence, numerous works were established in the human facial expression recognition system. Meanwhile, the detection and classification of micro-expressions are attracting attentions from various research communities in the recent few years. In this paper, we first review the processes of a conventional optical-flow-based recognition system, which comprised of facial landmarks annotations, optical flow guided images computation, features extraction and emotion class categorization. Secondly, a few approaches have been proposed to improve the feature extraction part, such as exploiting GAN to generate more image samples. Particularly, several variations of optical flow are computed in order to generate optimal images to lead to high recognition accuracy. Next, GAN, a combination of Generator and Discriminator, is utilized to generate new "fake" images to increase the sample size. Thirdly, a modified state-of-the-art Convolutional neural networks is proposed. To verify the effectiveness of the the proposed method, the results are evaluated on spontaneous micro-expression databases, namely SMIC, CASME II and SAMM. Both the F1-score and accuracy performance metrics are reported in this paper.