MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task

TL;DR

This paper introduces MUSER, a transformer-based multimodal model that leverages emotion recognition as an auxiliary task to enhance human stress detection, achieving state-of-the-art results on the MuSE dataset.

Contribution

The paper proposes a novel multi-task learning framework with a dynamic sampling strategy that improves stress detection by incorporating emotion recognition as an auxiliary task.

Findings

MUSER outperforms existing methods on the MuSE dataset.

Multi-task learning with emotion recognition enhances stress detection accuracy.

The dynamic sampling strategy improves training efficiency and model performance.

Abstract

The capability to automatically detect human stress can benefit artificial intelligent agents involved in affective computing and human-computer interaction. Stress and emotion are both human affective states, and stress has proven to have important implications on the regulation and expression of emotion. Although a series of methods have been established for multimodal stress detection, limited steps have been taken to explore the underlying inter-dependence between stress and emotion. In this work, we investigate the value of emotion recognition as an auxiliary task to improve stress detection. We propose MUSER -- a transformer-based model architecture and a novel multi-task learning algorithm with speed-based dynamic sampling strategy. Evaluations on the Multimodal Stressed Emotion (MuSE) dataset show that our model is effective for stress detection with both internal and external auxiliary tasks, and achieves state-of-the-art results.