From Data-Fitting to Discovery: Interpreting the Neural Dynamics of Motor Control through Reinforcement Learning

TL;DR

This paper investigates neural dynamics in reinforcement learning models of legged locomotion, revealing structured, less tangled neural trajectories that align with biological findings in primate movement.

Contribution

It demonstrates that embodied agents trained for walking exhibit neural dynamics similar to biological systems, bridging the gap between data-fitting models and biological plausibility.

Findings

Neural trajectories in the agent are less tangled in recurrent layers.

Identified speed axes that maximize variance in neural activity.

Structured neural activity supports experimental primate movement data.

Abstract

In motor neuroscience, artificial recurrent neural networks models often complement animal studies. However, most modeling efforts are limited to data-fitting, and the few that examine virtual embodied agents in a reinforcement learning context, do not draw direct comparisons to their biological counterparts. Our study addressing this gap, by uncovering structured neural activity of a virtual robot performing legged locomotion that directly support experimental findings of primate walking and cycling. We find that embodied agents trained to walk exhibit smooth dynamics that avoid tangling -- or opposing neural trajectories in neighboring neural space -- a core principle in computational neuroscience. Specifically, across a wide suite of gaits, the agent displays neural trajectories in the recurrent layers are less tangled than those in the input-driven actuation layers. To better interpret the neural separation of these elliptical-shaped trajectories, we identify speed axes that maximizes variance of mean activity across different forward, lateral, and rotational speed conditions.