A good body is all you need: avoiding catastrophic interference via agent architecture search

TL;DR

This paper demonstrates that optimizing a robot's physical structure, particularly sensor placement, can reduce catastrophic interference in policy learning, leading to more sample-efficient training and robust robot designs across environments.

Contribution

It introduces the concept of co-optimizing physical robot structure and control policies to mitigate catastrophic interference, a novel approach in robotics.

Findings

Physical structure influences catastrophic interference levels.

Co-optimization yields structures resistant to interference.

Sensor homeostasis across environments aids in overcoming interference.

Abstract

In robotics, catastrophic interference continues to restrain policy training across environments. Efforts to combat catastrophic interference to date focus on novel neural architectures or training methods, with a recent emphasis on policies with good initial settings that facilitate training in new environments. However, none of these methods to date have taken into account how the physical architecture of the robot can obstruct or facilitate catastrophic interference, just as the choice of neural architecture can. In previous work we have shown how aspects of a robot's physical structure (specifically, sensor placement) can facilitate policy learning by increasing the fraction of optimal policies for a given physical structure. Here we show for the first time that this proxy measure of catastrophic interference correlates with sample efficiency across several search methods, proving that favorable loss landscapes can be induced by the correct choice of physical structure. We show that such structures can be found via co-optimization -- optimization of a robot's structure and control policy simultaneously -- yielding catastrophic interference resistant robot structures and policies, and that this is more efficient than control policy optimization alone. Finally, we show that such structures exhibit sensor homeostasis across environments and introduce this as the mechanism by which certain robots overcome catastrophic interference.