Emotion Detection through Body Gesture and Face

TL;DR

This paper presents a novel approach to emotion recognition that uses body gestures and facial cues, leveraging deep learning and pose estimation to improve understanding of human emotions beyond facial analysis.

Contribution

It introduces a multimodal emotion detection system utilizing body gestures and facial cues, trained on large datasets, with a bifurcated neural network architecture for classification and regression tasks.

Findings

Achieved accurate recognition of seven basic emotions.

Successfully estimated valence and arousal scales.

Enhanced emotion detection by integrating body language with facial cues.

Abstract

The project leverages advanced machine and deep learning techniques to address the challenge of emotion recognition by focusing on non-facial cues, specifically hands, body gestures, and gestures. Traditional emotion recognition systems mainly rely on facial expression analysis and often ignore the rich emotional information conveyed through body language. To bridge this gap, this method leverages the Aff-Wild2 and DFEW databases to train and evaluate a model capable of recognizing seven basic emotions (angry, disgust, fear, happiness, sadness, surprise, and neutral) and estimating valence and continuous scales wakeup descriptor. Leverage OpenPose for pose estimation to extract detailed body posture and posture features from images and videos. These features serve as input to state-of-the-art neural network architectures, including ResNet, and ANN for emotion classification, and fully connected layers for valence arousal regression analysis. This bifurcation strategy can solve classification and regression problems in the field of emotion recognition. The project aims to contribute to the field of affective computing by enhancing the ability of machines to interpret and respond to human emotions in a more comprehensive and nuanced way. By integrating multimodal data and cutting-edge computational models, I aspire to develop a system that not only enriches human-computer interaction but also has potential applications in areas as diverse as mental health support, educational technology, and autonomous vehicle systems.