Affect and Effect: Limitations of regularisation-based continual learning in EEG-based emotion classification

TL;DR

This paper critically evaluates regularisation-based continual learning methods for EEG-based emotion classification, revealing their limitations in generalising to unseen subjects due to stability-plasticity trade-offs and data variability.

Contribution

It provides a theoretical and empirical analysis showing the limitations of regularisation-based CL methods in EEG emotion classification and highlights issues with parameter importance heuristics and data variability.

Findings

Regularisation-based CL methods perform poorly on DREAMER and SEED datasets.

Parameter importance heuristics become unreliable with noisy data.

High variability in EEG signals limits the benefit of past data for future subjects.

Abstract

Generalisation to unseen subjects in EEG-based emotion classification remains a challenge due to high inter-and intra-subject variability. Continual learning (CL) poses a promising solution by learning from a sequence of tasks while mitigating catastrophic forgetting. Regularisation-based CL approaches, such as Elastic Weight Consolidation (EWC), Synaptic Intelligence (SI), and Memory Aware Synapses (MAS), are commonly used as baselines in EEG-based CL studies, yet their suitability for this problem remains underexplored. This study theoretically and empirically finds that regularisation-based CL methods show limited performance for EEG-based emotion classification on the DREAMER and SEED datasets. We identify a fundamental misalignment in the stability-plasticity trade-off, where regularisation-based methods prioritise mitigating catastrophic forgetting (backward transfer) over adapting to new subjects (forward transfer). We investigate this limitation under subject-incremental sequences and observe that: (1) the heuristics for estimating parameter importance become less reliable under noisy data and covariate shift, (2) gradients on parameters deemed important by these heuristics often interfere with gradient updates required for new subjects, moving optimisation away from the minimum, (3) importance values accumulated across tasks over-constrain the model, and (4) performance is sensitive to subject order. Forward transfer showed no statistically significant improvement over sequential fine-tuning (p > 0.05 across approaches and datasets). The high variability of EEG signals means past subjects provide limited value to future subjects. Regularisation-based continual learning approaches are therefore limited for robust generalisation to unseen subjects in EEG-based emotion classification.