EEG-Based Multimodal Learning via Hyperbolic Mixture-of-Curvature Experts

TL;DR

This paper introduces EEG-MoCE, a hyperbolic mixture-of-curvature experts framework that models hierarchical structures in multimodal EEG data, significantly improving mental state assessment tasks.

Contribution

It proposes a novel hyperbolic mixture-of-curvature experts model that adaptively captures the intrinsic geometry of multimodal EEG data for enhanced learning.

Findings

Achieves state-of-the-art results on benchmark datasets for emotion recognition, sleep staging, and cognitive assessment.

Effectively models hierarchical structures in EEG and related modalities using learnable-curvature hyperbolic spaces.

Demonstrates the superiority of hyperbolic embeddings over Euclidean ones for multimodal neurotechnology tasks.

Abstract

Electroencephalography (EEG)-based multimodal learning integrates brain signals with complementary modalities to improve mental state assessment, providing great clinical potential. The effectiveness of such paradigms largely depends on the representation learning on heterogeneous modalities. For EEG-based paradigms, one promising approach is to leverage their hierarchical structures, as recent studies have shown that both EEG and associated modalities (e.g., facial expressions) exhibit hierarchical structures reflecting complex cognitive processes. However, Euclidean embeddings struggle to represent these hierarchical structures due to their flat geometry, while hyperbolic spaces, with their exponential growth property, are naturally suited for them. In this work, we propose EEG-MoCE, a novel hyperbolic mixture-of-curvature experts framework designed for multimodal neurotechnology. EEG-MoCE assigns each modality to an expert in a learnable-curvature hyperbolic space, enabling adaptive modeling of its intrinsic geometry. A curvature-aware fusion strategy then dynamically weights experts, emphasizing modalities with richer hierarchical information. Extensive experiments on benchmark datasets demonstrate that EEG-MoCE achieves state-of-the-art performance, including emotion recognition, sleep staging, and cognitive assessment.