Emotion Recognition with Pre-Trained Transformers Using Multimodal Signals

TL;DR

This paper explores using Transformer models pretrained on multimodal physiological signals to improve emotion recognition accuracy, demonstrating the effectiveness of pretraining and multimodal inputs on a state-of-the-art dataset.

Contribution

It introduces a Transformer-based multimodal emotion recognition framework with pretraining, advancing the state-of-the-art in physiological signal analysis.

Findings

Pretraining enhances emotion recognition performance.

Multimodal inputs outperform unimodal signals.

Transformer models are effective for physiological data analysis.

Abstract

In this paper, we address the problem of multimodal emotion recognition from multiple physiological signals. We demonstrate that a Transformer-based approach is suitable for this task. In addition, we present how such models may be pretrained in a multimodal scenario to improve emotion recognition performances. We evaluate the benefits of using multimodal inputs and pre-training with our approach on a state-ofthe-art dataset.