Reinforcement learning for freeform robot design

TL;DR

This paper introduces a reinforcement learning approach that enables the design of freeform robot morphologies with arbitrary structures by manipulating atomic building blocks, advancing beyond previous methods limited to simple limb adjustments.

Contribution

It presents a novel policy gradient method for designing complex robot structures with nonparametric macrostructures, expanding the scope of robot morphology optimization.

Findings

Successfully designed freeform robot structures using policy gradients.

Demonstrated potential for future closed-loop control and sim2real transfer.

Extended the capabilities of reinforcement learning in robot design.

Abstract

Inspired by the necessity of morphological adaptation in animals, a growing body of work has attempted to expand robot training to encompass physical aspects of a robot's design. However, reinforcement learning methods capable of optimizing the 3D morphology of a robot have been restricted to reorienting or resizing the limbs of a predetermined and static topological genus. Here we show policy gradients for designing freeform robots with arbitrary external and internal structure. This is achieved through actions that deposit or remove bundles of atomic building blocks to form higher-level nonparametric macrostructures such as appendages, organs and cavities. Although results are provided for open loop control only, we discuss how this method could be adapted for closed loop control and sim2real transfer to physical machines in future.