Efficient Morphology-Control Co-Design via Stackelberg Proximal Policy Optimization

TL;DR

This paper introduces Stackelberg PPO, a novel co-design method that models morphology-control coupling as a Stackelberg game, leading to more stable and efficient optimization in robotics design.

Contribution

It presents a new game-theoretic approach for morphology-control co-design, explicitly modeling control adaptation dynamics to improve optimization stability and efficiency.

Findings

Stackelberg PPO outperforms standard PPO in stability.

The method achieves higher final performance.

It enables more efficient robotics designs.

Abstract

Morphology-control co-design concerns the coupled optimization of an agent's body structure and control policy. This problem exhibits a bi-level structure, where the control dynamically adapts to the morphology to maximize performance. Existing methods typically neglect the control's adaptation dynamics by adopting a single-level formulation that treats the control policy as fixed when optimizing morphology. This can lead to inefficient optimization, as morphology updates may be misaligned with control adaptation. In this paper, we revisit the co-design problem from a game-theoretic perspective, modeling the intrinsic coupling between morphology and control as a novel variant of a Stackelberg game. We propose Stackelberg Proximal Policy Optimization (Stackelberg PPO), which explicitly incorporates the control's adaptation dynamics into morphology optimization. By modeling this intrinsic coupling, our method aligns morphology updates with control adaptation, thereby stabilizing training and improving learning efficiency. Experiments across diverse co-design tasks demonstrate that Stackelberg PPO outperforms standard PPO in both stability and final performance, opening the way for dramatically more efficient robotics designs.