Decentralized Deep Reinforcement Learning for a Distributed and Adaptive Locomotion Controller of a Hexapod Robot

TL;DR

This paper introduces a decentralized deep reinforcement learning approach for controlling a hexapod robot's locomotion, inspired by insect motor control, demonstrating improved learning speed and walking behavior.

Contribution

It presents a novel decentralized architecture for leg coordination in hexapod robots, learned via deep reinforcement learning, inspired by biological principles.

Findings

Decentralized structure learns better walking behavior.

Simpler organization learns faster than holistic approaches.

Decentralized approach improves robustness in locomotion control.

Abstract

Locomotion is a prime example for adaptive behavior in animals and biological control principles have inspired control architectures for legged robots. While machine learning has been successfully applied to many tasks in recent years, Deep Reinforcement Learning approaches still appear to struggle when applied to real world robots in continuous control tasks and in particular do not appear as robust solutions that can handle uncertainties well. Therefore, there is a new interest in incorporating biological principles into such learning architectures. While inducing a hierarchical organization as found in motor control has shown already some success, we here propose a decentralized organization as found in insect motor control for coordination of different legs. A decentralized and distributed architecture is introduced on a simulated hexapod robot and the details of the controller are learned through Deep Reinforcement Learning. We first show that such a concurrent local structure is able to learn better walking behavior. Secondly, that the simpler organization is learned faster compared to holistic approaches.