Learning Modular Robot Visual-motor Locomotion Policies

TL;DR

This paper introduces a modular visual-motor control policy for reconfigurable robots that generalizes across different designs and environments, enabling effective locomotion in challenging terrains.

Contribution

It develops a scalable deep reinforcement learning approach for training modular policies that adapt to various robot configurations and environments using visual input.

Findings

Policy successfully controls real robots climbing stairs and curbs.

Modular policy components can be recombined to handle different robot designs.

The approach generalizes to new configurations and environments during training.

Abstract

Control policy learning for modular robot locomotion has previously been limited to proprioceptive feedback and flat terrain. This paper develops policies for modular systems with vision traversing more challenging environments. These modular robots can be reconfigured to form many different designs, where each design needs a controller to function. Though one could create a policy for individual designs and environments, such an approach is not scalable given the wide range of potential designs and environments. To address this challenge, we create a visual-motor policy that can generalize to both new designs and environments. The policy itself is modular, in that it is divided into components, each of which corresponds to a type of module (e.g., a leg, wheel, or body). The policy components can be recombined during training to learn to control multiple designs. We develop a deep reinforcement learning algorithm where visual observations are input to a modular policy interacting with multiple environments at once. We apply this algorithm to train robots with combinations of legs and wheels, then demonstrate the policy controlling real robots climbing stairs and curbs.