Self-Relation Attention and Temporal Awareness for Emotion Recognition via Vocal Burst

TL;DR

This paper introduces a novel emotion recognition pipeline using self-relation attention and temporal awareness on vocal bursts, achieving significant improvements over baseline models in the A-VB 2022 challenge.

Contribution

The paper proposes a new method combining self-supervised feature extraction with self-relation attention and temporal awareness modules for emotion recognition from vocal bursts.

Findings

Achieved a mean CCC of 0.7295 on the test set.

Outperformed baseline model with a CCC of 0.5686.

Demonstrated effectiveness of the proposed modules in emotion prediction.

Abstract

The technical report presents our emotion recognition pipeline for high-dimensional emotion task (A-VB High) in The ACII Affective Vocal Bursts (A-VB) 2022 Workshop \& Competition. Our proposed method contains three stages. Firstly, we extract the latent features from the raw audio signal and its Mel-spectrogram by self-supervised learning methods. Then, the features from the raw signal are fed to the self-relation attention and temporal awareness (SA-TA) module for learning the valuable information between these latent features. Finally, we concatenate all the features and utilize a fully-connected layer to predict each emotion's score. By empirical experiments, our proposed method achieves a mean concordance correlation coefficient (CCC) of 0.7295 on the test set, compared to 0.5686 on the baseline model. The code of our method is available at https://github.com/linhtd812/A-VB2022.