Modulated Fusion using Transformer for Linguistic-Acoustic Emotion Recognition

TL;DR

This paper introduces two Transformer-based modulation architectures for linguistic-acoustic emotion recognition, achieving competitive results across multiple datasets with an emphasis on lightweight design and open-source implementation.

Contribution

Proposes novel lightweight Transformer architectures with modulation for emotion recognition, combining linguistic and acoustic data to surpass state-of-the-art performance.

Findings

Effective on IEMOCAP, MOSI, MOSEI, MELD datasets

Models outperform existing methods in emotion recognition

Open-source code facilitates future research

Abstract

This paper aims to bring a new lightweight yet powerful solution for the task of Emotion Recognition and Sentiment Analysis. Our motivation is to propose two architectures based on Transformers and modulation that combine the linguistic and acoustic inputs from a wide range of datasets to challenge, and sometimes surpass, the state-of-the-art in the field. To demonstrate the efficiency of our models, we carefully evaluate their performances on the IEMOCAP, MOSI, MOSEI and MELD dataset. The experiments can be directly replicated and the code is fully open for future researches.