Facial Expression Recognition Using a Hybrid CNN-SIFT Aggregator

TL;DR

This paper introduces a hybrid CNN-SIFT model for facial expression recognition that performs well with limited training data, achieving state-of-the-art accuracy on FER-2013 and CK+ datasets.

Contribution

It proposes a novel CNN-SIFT aggregation method that enhances facial expression recognition accuracy, especially with small datasets.

Findings

CNN with Dense SIFT outperforms conventional CNN and CNN with SIFT

Model aggregation improves accuracy further

Achieved state-of-the-art results on FER-2013 and CK+ datasets

Abstract

Deriving an effective facial expression recognition component is important for a successful human-computer interaction system. Nonetheless, recognizing facial expression remains a challenging task. This paper describes a novel approach towards facial expression recognition task. The proposed method is motivated by the success of Convolutional Neural Networks (CNN) on the face recognition problem. Unlike other works, we focus on achieving good accuracy while requiring only a small sample data for training. Scale Invariant Feature Transform (SIFT) features are used to increase the performance on small data as SIFT does not require extensive training data to generate useful features. In this paper, both Dense SIFT and regular SIFT are studied and compared when merged with CNN features. Moreover, an aggregator of the models is developed. The proposed approach is tested on the FER-2013 and CK+ datasets. Results demonstrate the superiority of CNN with Dense SIFT over conventional CNN and CNN with SIFT. The accuracy even increased when all the models are aggregated which generates state-of-art results on FER-2013 and CK+ datasets, where it achieved 73.4% on FER-2013 and 99.1% on CK+.