It's LeVAsa not LevioSA! Latent Encodings for Valence-Arousal Structure Alignment

TL;DR

This paper introduces LeVAsa, a variational autoencoder that aligns its latent space with Valence-Arousal emotion dimensions, improving emotion prediction and interpretability in affective computing.

Contribution

It presents a novel algorithm for mapping categorical and dimensional emotion labels and a VAE model that aligns its latent space with VA space for better emotion representation.

Findings

LeVAsa achieves high latent-circumplex alignment.

Improved categorical emotion prediction performance.

Trade-off observed between alignment degree and reconstruction quality.

Abstract

In recent years, great strides have been made in the field of affective computing. Several models have been developed to represent and quantify emotions. Two popular ones include (i) categorical models which represent emotions as discrete labels, and (ii) dimensional models which represent emotions in a Valence-Arousal (VA) circumplex domain. However, there is no standard for annotation mapping between the two labelling methods. We build a novel algorithm for mapping categorical and dimensional model labels using annotation transfer across affective facial image datasets. Further, we utilize the transferred annotations to learn rich and interpretable data representations using a variational autoencoder (VAE). We present "LeVAsa", a VAE model that learns implicit structure by aligning the latent space with the VA space. We evaluate the efficacy of LeVAsa by comparing performance with the Vanilla VAE using quantitative and qualitative analysis on two benchmark affective image datasets. Our results reveal that LeVAsa achieves high latent-circumplex alignment which leads to improved downstream categorical emotion prediction. The work also demonstrates the trade-off between degree of alignment and quality of reconstructions.