Learning Unseen Emotions from Gestures via Semantically-Conditioned Zero-Shot Perception with Adversarial Autoencoders

TL;DR

This paper introduces a zero-shot learning approach using adversarial autoencoders to recognize unseen emotions from gestures by correlating gesture sequences with semantic emotion embeddings, achieving significant accuracy improvements.

Contribution

The paper proposes a novel adversarial autoencoder-based method that maps gesture sequences to semantic emotion labels in a zero-shot setting, improving upon existing algorithms.

Findings

Achieved 58.43% accuracy on MPI EBEDB

Improved state-of-the-art zero-shot emotion recognition by 25-27%

Demonstrated effective mapping of gestures to unseen emotion categories

Abstract

We present a novel generalized zero-shot algorithm to recognize perceived emotions from gestures. Our task is to map gestures to novel emotion categories not encountered in training. We introduce an adversarial, autoencoder-based representation learning that correlates 3D motion-captured gesture sequence with the vectorized representation of the natural-language perceived emotion terms using word2vec embeddings. The language-semantic embedding provides a representation of the emotion label space, and we leverage this underlying distribution to map the gesture-sequences to the appropriate categorical emotion labels. We train our method using a combination of gestures annotated with known emotion terms and gestures not annotated with any emotions. We evaluate our method on the MPI Emotional Body Expressions Database (EBEDB) and obtain an accuracy of $58.43\%$. This improves the performance of current state-of-the-art algorithms for generalized zero-shot learning by $25$--$27\%$ on the absolute.