Facial Expression Recognition using Facial Landmark Detection and Feature Extraction via Neural Networks

TL;DR

This paper presents a framework for facial expression recognition that combines facial landmark detection with feature extraction and neural network classification, achieving improved recognition of universal emotions.

Contribution

It introduces a hybrid approach using landmark detection and traditional feature extraction methods with neural networks for emotion classification.

Findings

Achieved higher uniformity in recognizing certain emotions.

Addressed the subjective nature of expression recognition.

Utilized both state-of-the-art and traditional detection algorithms.

Abstract

The proposed framework in this paper has the primary objective of classifying the facial expression shown by a person. These classifiable expressions can be any one of the six universal emotions along with the neutral emotion. After the initial facial localization is performed, facial landmark detection and feature extraction are applied where in the landmarks are determined to be the fiducial features: the eyebrows, eyes, nose and lips. This is primarily done using state-of-the-art facial landmark detection algorithms as well as traditional edge and corner point detection methods using Sobel filters and Shi Tomasi corner point detection methods respectively. This leads to generation of input feature vectors being formulated using Euclidean distances and trained into a Multi-Layer Perceptron (MLP) neural network in order to classify the expression being displayed. The results achieved have further dealt with higher uniformity in certain emotions and the inherently subjective nature of expression.