Cross-Modal Consistency-Guided Active Learning for Affective BCI Systems

TL;DR

This paper introduces a novel active learning framework for EEG-based emotion recognition that uses cross-modal consistency to improve robustness against noisy labels and artifacts, enhancing data efficiency and reliability.

Contribution

It proposes a cross-modal consistency-guided active learning method that jointly models uncertainty and semantic coherence between EEG and face data for affective BCI systems.

Findings

Improves robustness to label noise in EEG emotion recognition

Enhances data efficiency by selective querying based on cross-modal consistency

Demonstrates superior performance on the ASCERTAIN dataset

Abstract

Deep learning models perform best with abundant, high-quality labels, yet such conditions are rarely achievable in EEG-based emotion recognition. Electroencephalogram (EEG) signals are easily corrupted by artifacts and individual variability, while emotional labels often stem from subjective and inconsistent reports-making robust affective decoding particularly difficult. We propose an uncertainty-aware active learning framework that enhances robustness to label noise by jointly leveraging model uncertainty and cross-modal consistency. Instead of relying solely on EEG-based uncertainty estimates, the method evaluates cross-modal alignment to determine whether uncertainty originates from cognitive ambiguity or sensor noise. A representation alignment module embeds EEG and face features into a shared latent space, enforcing semantic coherence between modalities. Residual discrepancies are treated as noise-induced inconsistencies, and these samples are selectively queried for oracle feedback during active learning. This feedback-driven process guides the network toward reliable, informative samples and reduces the impact of noisy labels. Experiments on the ASCERTAIN dataset examine the efficiency and robustness of ours, highlighting its potential as a data-efficient and noise-tolerant approach for EEG-based affective decoding in brain-computer interface systems.