Bio-inspired Learning of Sensorimotor Control for Locomotion

TL;DR

This paper introduces a bio-inspired control architecture using coupled oscillators and Q-learning to optimize snake robot locomotion and maneuvering, demonstrating effective control in simulation.

Contribution

It presents a novel CPG-based architecture combined with Q-learning for learning optimal locomotion control in snake robots.

Findings

Effective maneuvering control achieved in simulation

Coupled oscillator feedback particle filter approximates posterior distributions

Q-learning successfully learns optimal control laws

Abstract

This paper presents a bio-inspired central pattern generator (CPG)-type architecture for learning optimal maneuvering control of periodic locomotory gaits. The architecture is presented here with the aid of a snake robot model problem involving planar locomotion of coupled rigid body systems. The maneuver involves clockwise or counterclockwise turning from a nominally straight path. The CPG circuit is realized as a coupled oscillator feedback particle filter. The collective dynamics of the filter are used to approximate a posterior distribution that is used to construct the optimal control input for maneuvering the robot. A Q-learning algorithm is applied to learn the approximate optimal control law. The issues surrounding the parametrization of the Q-function are discussed. The theoretical results are illustrated with numerics for a 5-link snake robot system.