Data-efficient Co-Adaptation of Morphology and Behaviour with Deep Reinforcement Learning

TL;DR

This paper introduces a data-efficient deep reinforcement learning method for automatically co-adapting robot morphology and control, leveraging prior knowledge to reduce the number of prototypes needed for optimal design.

Contribution

It presents a novel approach that uses previous morphologies and behaviors to inform and accelerate the co-adaptation process in robotics.

Findings

Requires fewer prototypes to find effective morphology-behavior combinations

Leverages past tested configurations for informed decision-making

Suitable for real-world robot design co-adaptation

Abstract

Humans and animals are capable of quickly learning new behaviours to solve new tasks. Yet, we often forget that they also rely on a highly specialized morphology that co-adapted with motor control throughout thousands of years. Although compelling, the idea of co-adapting morphology and behaviours in robots is often unfeasible because of the long manufacturing times, and the need to re-design an appropriate controller for each morphology. In this paper, we propose a novel approach to automatically and efficiently co-adapt a robot morphology and its controller. Our approach is based on recent advances in deep reinforcement learning, and specifically the soft actor critic algorithm. Key to our approach is the possibility of leveraging previously tested morphologies and behaviors to estimate the performance of new candidate morphologies. As such, we can make full use of the information available for making more informed decisions, with the ultimate goal of achieving a more data-efficient co-adaptation (i.e., reducing the number of morphologies and behaviors tested). Simulated experiments show that our approach requires drastically less design prototypes to find good morphology-behaviour combinations, making this method particularly suitable for future co-adaptation of robot designs in the real world.