EEG-Based Brain-LLM Interface for Human Preference Aligned Generation

TL;DR

This paper explores using EEG neural signals as an alternative input method for large language models to support users with speech or motor impairments, enabling real-time preference-based adaptation.

Contribution

It introduces a brain-LLM interface that predicts user satisfaction from EEG signals and integrates this feedback into model inference, pioneering neural feedback integration in LLMs.

Findings

EEG signals can predict user satisfaction with LLM outputs.

Neural feedback can be used to adapt LLM inference dynamically.

The approach supports users with speech or motor impairments.

Abstract

Large language models (LLMs) are becoming an increasingly important component of human--computer interaction, enabling users to coordinate a wide range of intelligent agents through natural language. While language-based interfaces are powerful and flexible, they implicitly assume that users can reliably produce explicit linguistic input, an assumption that may not hold for users with speech or motor impairments, e.g., Amyotrophic Lateral Sclerosis (ALS). In this work, we investigate whether neural signals can be used as an alternative input to LLMs, particularly to support those socially marginalized or underserved users. We build a simple brain-LLM interface, which uses EEG signals to guide image generation models at test time. Specifically, we first train a classifier to estimate user satisfaction from EEG signals. Its predictions are then incorporated into a test-time scaling (TTS) framework that dynamically adapts model inference using neural feedback collected during user evaluation. The experiments show that EEG can predict user satisfaction, suggesting that neural activity carries information on real-time preference inference. These findings provide a first step toward integrating neural feedback into adaptive language-model inference, and hopefully open up new possibilities for future research on adaptive LLM interaction.