Growable and Interpretable Neural Control with Online Continual Learning for Autonomous Lifelong Locomotion Learning Machines

TL;DR

This paper introduces GOLLUM, an interpretable neural control framework enabling autonomous, continual locomotion skill learning on robots, effectively addressing catastrophic forgetting, sample inefficiency, and knowledge exploitation without human intervention.

Contribution

GOLLUM is a novel, interpretable, growable neural control system that facilitates autonomous lifelong learning of robot locomotion skills through online continual learning and neurogenesis.

Findings

Successfully learned multiple locomotion skills on a hexapod robot within an hour.

Effectively transferred and combined learned skills to facilitate new skill acquisition.

Prevented catastrophic forgetting during continual learning process.

Abstract

Continual locomotion learning faces four challenges: incomprehensibility, sample inefficiency, lack of knowledge exploitation, and catastrophic forgetting. Thus, this work introduces Growable Online Locomotion Learning Under Multicondition (GOLLUM), which exploits the interpretability feature to address the aforementioned challenges. GOLLUM has two dimensions of interpretability: layer-wise interpretability for neural control function encoding and column-wise interpretability for robot skill encoding. With this interpretable control structure, GOLLUM utilizes neurogenesis to unsupervisely increment columns (ring-like networks); each column is trained separately to encode and maintain a specific primary robot skill. GOLLUM also transfers the parameters to new skills and supplements the learned combination of acquired skills through another neural mapping layer added (layer-wise) with online supplementary learning. On a physical hexapod robot, GOLLUM successfully acquired multiple locomotion skills (e.g., walking, slope climbing, and bouncing) autonomously and continuously within an hour using a simple reward function. Furthermore, it demonstrated the capability of combining previous learned skills to facilitate the learning process of new skills while preventing catastrophic forgetting. Compared to state-of-the-art locomotion learning approaches, GOLLUM is the only approach that addresses the four challenges above mentioned without human intervention. It also emphasizes the potential exploitation of interpretability to achieve autonomous lifelong learning machines.