Worm-level Control through Search-based Reinforcement Learning

TL;DR

This paper demonstrates how biological neural circuit models, specifically the C. elegans tap-withdrawal circuit, can be repurposed using search-based reinforcement learning to control a dynamic system, achieving performance comparable to traditional methods.

Contribution

It introduces a novel approach of re-purposing biological neural circuits with search-based reinforcement learning for control tasks.

Findings

Neural circuit-based policies perform as well as traditional control methods.

Re-purposing biological circuits is effective for control applications.

The approach is demonstrated on the inverted pendulum problem.

Abstract

Through natural evolution, nervous systems of organisms formed near-optimal structures to express behavior. Here, we propose an effective way to create control agents, by \textit{re-purposing} the function of biological neural circuit models, to govern similar real world applications. We model the tap-withdrawal (TW) neural circuit of the nematode, \textit{C. elegans}, a circuit responsible for the worm's reflexive response to external mechanical touch stimulations, and learn its synaptic and neural parameters as a policy for controlling the inverted pendulum problem. For reconfiguration of the purpose of the TW neural circuit, we manipulate a search-based reinforcement learning. We show that our neural policy performs as good as existing traditional control theory and machine learning approaches. A video demonstration of the performance of our method can be accessed at \url{https://youtu.be/o-Ia5IVyff8}.