Autonomous Reinforcement of Behavioral Sequences in Neural Dynamics

TL;DR

This paper presents DN-SARSA(λ), a neural algorithm that learns behavioral sequences from delayed rewards, integrating neural models, reinforcement learning, and working memory, validated on robots to demonstrate neural-compatible learning.

Contribution

It introduces DN-SARSA(λ), a novel neural reinforcement learning algorithm combining neural dynamics with sequence learning, validated on robotic platforms.

Findings

DN-SARSA(λ) performs comparably to discrete SARSA(λ) in learning sequences.

The algorithm successfully learns from exploration in both simulated and real robots.

Demonstrates feasibility of neural-compatible reinforcement learning for behavioral sequences.

Abstract

We introduce a dynamic neural algorithm called Dynamic Neural (DN) SARSA(\lambda) for learning a behavioral sequence from delayed reward. DN-SARSA(\lambda) combines Dynamic Field Theory models of behavioral sequence representation, classical reinforcement learning, and a computational neuroscience model of working memory, called Item and Order working memory, which serves as an eligibility trace. DN-SARSA(\lambda) is implemented on both a simulated and real robot that must learn a specific rewarding sequence of elementary behaviors from exploration. Results show DN-SARSA(\lambda) performs on the level of the discrete SARSA(\lambda), validating the feasibility of general reinforcement learning without compromising neural dynamics.