Reinforcement Learning from Implicit Neural Feedback for Human-Aligned Robot Control

TL;DR

This paper introduces a novel reinforcement learning framework that leverages implicit EEG-based neural feedback to train robots effectively without explicit human input, improving learning efficiency and alignment.

Contribution

It presents a new RLIHF method that decodes EEG error signals into rewards, enabling human-aligned robot control without explicit feedback mechanisms.

Findings

Agents trained with EEG feedback perform comparably to those with manual rewards.

The approach reduces user cognitive load during training.

Effective policy learning is achieved with sparse external rewards.

Abstract

Conventional reinforcement learning (RL) approaches often struggle to learn effective policies under sparse reward conditions, necessitating the manual design of complex, task-specific reward functions. To address this limitation, reinforcement learning from human feedback (RLHF) has emerged as a promising strategy that complements hand-crafted rewards with human-derived evaluation signals. However, most existing RLHF methods depend on explicit feedback mechanisms such as button presses or preference labels, which disrupt the natural interaction process and impose a substantial cognitive load on the user. We propose a novel reinforcement learning from implicit human feedback (RLIHF) framework that utilizes non-invasive electroencephalography (EEG) signals, specifically error-related potentials (ErrPs), to provide continuous, implicit feedback without requiring explicit user intervention. The proposed method adopts a pre-trained decoder to transform raw EEG signals into probabilistic reward components, enabling effective policy learning even in the presence of sparse external rewards. We evaluate our approach in a simulation environment built on the MuJoCo physics engine, using a Kinova Gen2 robotic arm to perform a complex pick-and-place task that requires avoiding obstacles while manipulating target objects. The results show that agents trained with decoded EEG feedback achieve performance comparable to those trained with dense, manually designed rewards. These findings validate the potential of using implicit neural feedback for scalable and human-aligned reinforcement learning in interactive robotics.