Distinguishing Posed and Spontaneous Smiles by Facial Dynamics

TL;DR

This paper presents an automatic method for classifying posed and spontaneous smiles using deep learning, optical flow, and normalization techniques, achieving promising accuracy on a large smile database.

Contribution

It introduces a novel combination of CNN features, local phase quantization, optical flow, and normalization to improve smile classification accuracy.

Findings

HOG features outperform CNN face model in classification.

Normalization improves classification accuracy.

EVM micro-expression amplification has limited impact.

Abstract

Smile is one of the key elements in identifying emotions and present state of mind of an individual. In this work, we propose a cluster of approaches to classify posed and spontaneous smiles using deep convolutional neural network (CNN) face features, local phase quantization (LPQ), dense optical flow and histogram of gradient (HOG). Eulerian Video Magnification (EVM) is used for micro-expression smile amplification along with three normalization procedures for distinguishing posed and spontaneous smiles. Although the deep CNN face model is trained with large number of face images, HOG features outperforms this model for overall face smile classification task. Using EVM to amplify micro-expressions did not have a significant impact on classification accuracy, while the normalizing facial features improved classification accuracy. Unlike many manual or semi-automatic methodologies, our approach aims to automatically classify all smiles into either `spontaneous' or `posed' categories, by using support vector machines (SVM). Experimental results on large UvA-NEMO smile database show promising results as compared to other relevant methods.