ALIGN: Adversarial Learning for Generalizable Speech Neuroprosthesis

TL;DR

ALIGN introduces an adversarial learning framework that enhances the generalization of intracortical speech decoding models across different recording sessions, addressing session variability issues in brain-computer interfaces.

Contribution

The paper proposes a novel session-invariant learning method using multi-domain adversarial neural networks for semi-supervised cross-session adaptation in BCIs.

Findings

ALIGN improves phoneme and word error rates on unseen sessions.

Adversarial domain alignment effectively mitigates session-level distribution shifts.

The method enhances robustness of speech decoding over time.

Abstract

Intracortical brain-computer interfaces (BCIs) can decode speech from neural activity with high accuracy when trained on data pooled across recording sessions. In realistic deployment, however, models must generalize to new sessions without labeled data, and performance often degrades due to cross-session nonstationarities (e.g., electrode shifts, neural turnover, and changes in user strategy). In this paper, we propose ALIGN, a session-invariant learning framework based on multi-domain adversarial neural networks for semi-supervised cross-session adaptation. ALIGN trains a feature encoder jointly with a phoneme classifier and a domain classifier operating on the latent representation. Through adversarial optimization, the encoder is encouraged to preserve task-relevant information while suppressing session-specific cues. We evaluate ALIGN on intracortical speech decoding and find that it generalizes consistently better to previously unseen sessions, improving both phoneme error rate and word error rate relative to baselines. These results indicate that adversarial domain alignment is an effective approach for mitigating session-level distribution shift and enabling robust longitudinal BCI decoding.