Real-Time Facial Expression Recognition using Facial Landmarks and Neural Networks

TL;DR

This paper introduces a real-time facial expression recognition system using facial landmarks and neural networks, achieving high accuracy with a lightweight feature extraction method suitable for static images.

Contribution

The paper proposes a novel lightweight algorithm combining geometric and texture features with facial landmarks for real-time emotion classification.

Findings

Achieved 96% accuracy on test set

Effective real-time facial expression recognition

Combines geometric and texture features

Abstract

This paper presents a lightweight algorithm for feature extraction, classification of seven different emotions, and facial expression recognition in a real-time manner based on static images of the human face. In this regard, a Multi-Layer Perceptron (MLP) neural network is trained based on the foregoing algorithm. In order to classify human faces, first, some pre-processing is applied to the input image, which can localize and cut out faces from it. In the next step, a facial landmark detection library is used, which can detect the landmarks of each face. Then, the human face is split into upper and lower faces, which enables the extraction of the desired features from each part. In the proposed model, both geometric and texture-based feature types are taken into account. After the feature extraction phase, a normalized vector of features is created. A 3-layer MLP is trained using these feature vectors, leading to 96% accuracy on the test set.