Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels

TL;DR

This paper introduces CURSOR, a novel self-calibrating framework that recovers mental targets from EEG and image data without labeled training data, enabling ranking and generation of stimuli aligned with mental images.

Contribution

The paper presents the first self-calibrating method for recovering mental targets from EEG and images without labeled data or pre-trained decoders.

Findings

CURSOR predicts image similarity scores correlating with human judgments.

It ranks stimuli against an unknown mental target effectively.

It generates stimuli indistinguishable from the mental target, validated by user study.

Abstract

We consider the problem of recovering a mental target (e.g., an image of a face) that a participant has in mind from paired EEG (i.e., brain responses) and image (i.e., perceived faces) data collected during interactive sessions without access to labeled information. The problem has been previously explored with labeled data but not via self-calibration, where labeled data is unavailable. Here, we present the first framework and an algorithm, CURSOR, that learns to recover unknown mental targets without access to labeled data or pre-trained decoders. Our experiments on naturalistic images of faces demonstrate that CURSOR can (1) predict image similarity scores that correlate with human perceptual judgments without any label information, (2) use these scores to rank stimuli against an unknown mental target, and (3) generate new stimuli indistinguishable from the unknown mental target (validated via a user study, N=53).