Design strategies for controlling neuron-connected robots using reinforcement learning

TL;DR

This paper introduces design principles for training neuron-connected robots with reinforcement learning to achieve context-dependent behaviors aligned with task goals, using neural data-based simulators and adaptive training.

Contribution

It proposes a novel framework combining deep reinforcement learning with physiologically valid neural simulators that are updated during training for improved robot control.

Findings

Robots learned context-dependent behaviors in pole balancing and navigation tasks.

Policies remained valid across different neural data-based simulators.

Updating simulators during training improved task performance.

Abstract

Despite the growing interest in robot control utilizing the computation of biological neurons, context-dependent behavior by neuron-connected robots remains a challenge. Context-dependent behavior here is defined as behavior that is not the result of a simple sensory-motor coupling, but rather based on an understanding of the task goal. This paper proposes design principles for training neuron-connected robots based on task goals to achieve context-dependent behavior. First, we employ deep reinforcement learning (RL) to enable training that accounts for goal achievements. Second, we propose a neuron simulator as a probability distribution based on recorded neural data, aiming to represent physiologically valid neural dynamics while avoiding complex modeling with high computational costs. Furthermore, we propose to update the simulators during the training to bridge the gap between the simulation and the real settings. The experiments showed that the robot gradually learned context-dependent behaviors in pole balancing and robot navigation tasks. Moreover, the learned policies were valid for neural simulators based on novel neural data, and the task performance increased by updating the simulators during training. These results suggest the effectiveness of the proposed design principle for the context-dependent behavior of neuron-connected robots.