Accelerated co-design of robots through morphological pretraining

TL;DR

This paper introduces a method for rapid robot co-design using morphological pretraining and differentiable simulation, enabling quick evaluation of design changes and maintaining diversity in evolved robot populations.

Contribution

The authors present a novel morphological pretraining approach that allows zero-shot evaluation of robot designs and improves co-evolution efficiency over traditional methods.

Findings

Pretrained controllers enable rapid assessment of non-differentiable design changes.

Zero-shot evolution produces diverse, high-performing robot designs.

Fine-tuning pretrained controllers enhances diversity and performance during evolution.

Abstract

The co-design of robot morphology and neural control typically requires using reinforcement learning to approximate a unique control policy gradient for each body plan, demanding massive amounts of training data to measure the performance of each design. Here we show that a universal, morphology-agnostic controller can be rapidly and directly obtained by gradient-based optimization through differentiable simulation. This process of morphological pretraining allows the designer to explore non-differentiable changes to a robot's physical layout (e.g. adding, removing and recombining discrete body parts) and immediately determine which revisions are beneficial and which are deleterious using the pretrained model. We term this process "zero-shot evolution" and compare it with the simultaneous co-optimization of a universal controller alongside an evolving design population. We find the latter results in diversity collapse, a previously unknown pathology whereby the population -- and thus the controller's training data -- converges to similar designs that are easier to steer with a shared universal controller. We show that zero-shot evolution with a pretrained controller quickly yields a diversity of highly performant designs, and by fine-tuning the pretrained controller on the current population throughout evolution, diversity is not only preserved but significantly increased as superior performance is achieved.