Towards Interactive Reinforcement Learning with Intrinsic Feedback

TL;DR

This paper reviews the emerging concept of intrinsic feedback, derived from neural activity, as a natural and automatic form of human input for interactive reinforcement learning, aiming to enhance human-in-the-loop AI systems.

Contribution

It introduces the concept of intrinsic feedback from neural signals, connecting BCI and RL fields, and provides a tutorial review of its motivations, approaches, and open challenges.

Findings

Intrinsic feedback can be conveyed automatically and unconsciously.

The integration of neural signals as feedback enhances interactive RL.

Open problems include effective exploration and interpretation of neural feedback.

Abstract

Reinforcement learning (RL) and brain-computer interfaces (BCI) have experienced significant growth over the past decade. With rising interest in human-in-the-loop (HITL), incorporating human input with RL algorithms has given rise to the sub-field of interactive RL. Adjacently, the field of BCI has long been interested in extracting informative brain signals from neural activity for use in human-computer interactions. A key link between these fields lies in the interpretation of neural activity as feedback such that interactive RL approaches can be employed. We denote this new and emerging medium of feedback as intrinsic feedback. Despite intrinsic feedback's ability to be conveyed automatically and even unconsciously, proper exploration surrounding this key link has largely gone unaddressed by both communities. Thus, to help facilitate a deeper understanding and a more effective utilization, we provide a tutorial-style review covering the motivations, approaches, and open problems of intrinsic feedback and its foundational concepts.