A Dynamic 3D Spontaneous Micro-expression Database: Establishment and Evaluation

TL;DR

This paper introduces a new 3D micro-expression database with both 2D videos and 3D point clouds, demonstrating improved classification accuracy through 3D feature fusion, advancing research in spontaneous facial micro-expressions.

Contribution

The creation of a comprehensive 3D micro-expression database and evaluation of 3D features for improved classification accuracy over traditional 2D methods.

Findings

Fusion of 3D and 2D features improves classification accuracy.

3D features outperform 2D features in micro-expression recognition.

The database supports exploration of 3D spatio-temporal micro-expression features.

Abstract

Micro-expressions are spontaneous, unconscious facial movements that show people's true inner emotions and have great potential in related fields of psychological testing. Since the face is a 3D deformation object, the occurrence of an expression can arouse spatial deformation of the face, but limited by the available databases are 2D videos, lacking the description of 3D spatial information of micro-expressions. Therefore, we proposed a new micro-expression database containing 2D video sequences and 3D point clouds sequences. The database includes 373 micro-expressions sequences, and these samples were classified using the objective method based on facial action coding system, as well as the non-objective method that combines video contents and participants' self-reports. We extracted 2D and 3D features using the local binary patterns on three orthogonal planes (LBP-TOP) and curvature algorithms, respectively, and evaluated the classification accuracies of these two features and their fusion results with leave-one-subject-out (LOSO) and 10-fold cross-validation. Further, we performed various neural network algorithms for database classification, the results show that classification accuracies are improved by fusing 3D features than using only 2D features. The database offers original and cropped micro-expression samples, which will facilitate the exploration and research on 3D Spatio-temporal features of micro-expressions.