Multi-Masked Querying Network for Robust Emotion Recognition from Incomplete Multi-Modal Physiological Signals

TL;DR

This paper introduces MMQ-Net, a novel neural network that effectively handles incomplete multi-modal physiological signals and noise for improved emotion recognition, demonstrating superior performance over existing methods.

Contribution

The paper proposes a multi-masked querying network that integrates modality, category, and interference queries to enhance emotion recognition from incomplete and noisy physiological data.

Findings

Outperforms existing methods in emotion recognition accuracy.

Effectively reconstructs missing data in incomplete signals.

Robust against noise and artifacts in physiological signals.

Abstract

Emotion recognition from physiological data is crucial for mental health assessment, yet it faces two significant challenges: incomplete multi-modal signals and interference from body movements and artifacts. This paper presents a novel Multi-Masked Querying Network (MMQ-Net) to address these issues by integrating multiple querying mechanisms into a unified framework. Specifically, it uses modality queries to reconstruct missing data from incomplete signals, category queries to focus on emotional state features, and interference queries to separate relevant information from noise. Extensive experiment results demonstrate the superior emotion recognition performance of MMQ-Net compared to existing approaches, particularly under high levels of data incompleteness.