Inverse reinforcement learning conditioned on brain scan

TL;DR

This paper proposes a method for inverse reinforcement learning that uses fMRI brain scans as part of the state space to model human thought processes and train humanoid robots to act based on brain activity and environment.

Contribution

It introduces a novel approach combining brain scan data with inverse reinforcement learning to model human dispositions and control robots accordingly.

Findings

Model can predict brain states conditioned on environment

Robot actions are conditioned on fMRI-derived states

Generative model predicts future brain scans

Abstract

We outline a way for an agent to learn the dispositions of a particular individual through inverse reinforcement learning where the state space at time t includes an fMRI scan of the individual, to represent his brain state at that time. The fundamental assumption being that the information shown on an fMRI scan of an individual is conditioned on his thoughts and thought processes. The system models both long and short term memory as well any internal dynamics we may not be aware of that are in the human brain. The human expert will put on a suit for a set duration with sensors whose information will be used to train a policy network, while a generative model will be trained to produce the next fMRI scan image conditioned on the present one and the state of the environment. During operation the humanoid robots actions will be conditioned on this evolving fMRI and the environment it is in.