Emotion estimation from video footage with LSTM

TL;DR

This paper introduces an LSTM-based model that analyzes facial expressions from live video streams using MediaPipe blend-shapes to estimate emotions with high accuracy and reduced computational costs.

Contribution

The novel contribution is the application of an LSTM model to process MediaPipe blend-shapes for real-time emotion estimation from video footage.

Findings

Achieved 71% accuracy in emotion estimation

Attained 62% F1-score on FER2013 dataset

Reduced computational costs compared to existing methods

Abstract

Emotion estimation in general is a field that has been studied for a long time, and several approaches exist using machine learning. in this paper, we present an LSTM model, that processes the blend-shapes produced by the library MediaPipe, for a face detected in a live stream of a camera, to estimate the main emotion from the facial expressions, this model is trained on the FER2013 dataset and delivers a result of 71% accuracy and 62% f1-score which meets the accuracy benchmark of the FER2013 dataset, with significantly reduced computation costs. https://github.com/Samir-atra/Emotion_estimation_from_video_footage_with_LSTM_ML_algorithm