What happens in Face during a facial expression? Using data mining techniques to analyze facial expression motion vectors

TL;DR

This study employs optical flow and data mining techniques to analyze facial motion vectors for automatic facial expression recognition, achieving high accuracy with deep learning, SVM, and C5.0 classifiers on the CK+ dataset.

Contribution

It introduces a novel approach combining optical flow with various data mining algorithms for improved facial expression recognition.

Findings

Deep learning achieved 95.3% accuracy

SVM achieved 92.8% accuracy

C5.0 achieved 90.2% accuracy

Abstract

One of the most common problems encountered in human-computer interaction is automatic facial expression recognition. Although it is easy for human observer to recognize facial expressions, automatic recognition remains difficult for machines. One of the methods that machines can recognize facial expression is analyzing the changes in face during facial expression presentation. In this paper, optical flow algorithm was used to extract deformation or motion vectors created in the face because of facial expressions. Then, these extracted motion vectors are used to be analyzed. Their positions and directions were exploited for automatic facial expression recognition using different data mining techniques. It means that by employing motion vector features used as our data, facial expressions were recognized. Some of the most state-of-the-art classification algorithms such as C5.0, CRT, QUEST, CHAID, Deep Learning (DL), SVM and Discriminant algorithms were used to classify the extracted motion vectors. Using 10-fold cross validation, their performances were calculated. To compare their performance more precisely, the test was repeated 50 times. Meanwhile, the deformation of face was also analyzed in this research. For example, what exactly happened in each part of face when a person showed fear? Experimental results on Extended Cohen-Kanade (CK+) facial expression dataset demonstrated that the best methods were DL, SVM and C5.0, with the accuracy of 95.3%, 92.8% and 90.2% respectively.