Emotion Detection through Body Gesture and Face
Haoyang Liu

TL;DR
This paper presents a novel approach to emotion recognition that uses body gestures and facial cues, leveraging deep learning and pose estimation to improve understanding of human emotions beyond facial analysis.
Contribution
It introduces a multimodal emotion detection system utilizing body gestures and facial cues, trained on large datasets, with a bifurcated neural network architecture for classification and regression tasks.
Findings
Achieved accurate recognition of seven basic emotions.
Successfully estimated valence and arousal scales.
Enhanced emotion detection by integrating body language with facial cues.
Abstract
The project leverages advanced machine and deep learning techniques to address the challenge of emotion recognition by focusing on non-facial cues, specifically hands, body gestures, and gestures. Traditional emotion recognition systems mainly rely on facial expression analysis and often ignore the rich emotional information conveyed through body language. To bridge this gap, this method leverages the Aff-Wild2 and DFEW databases to train and evaluate a model capable of recognizing seven basic emotions (angry, disgust, fear, happiness, sadness, surprise, and neutral) and estimating valence and continuous scales wakeup descriptor. Leverage OpenPose for pose estimation to extract detailed body posture and posture features from images and videos. These features serve as input to state-of-the-art neural network architectures, including ResNet, and ANN for emotion classification, and fully…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems
MethodsAverage Pooling · Max Pooling · Global Average Pooling · Kaiming Initialization · Convolution · OpenPose
