Benefits of Low-Cost Bio-Inspiration in the Age of Overparametrization

TL;DR

This study compares bio-inspired control paradigms for robots, showing that simpler models with fewer parameters often outperform complex, overparameterized neural networks, especially when using evolutionary training methods.

Contribution

The paper provides empirical evidence that low-cost, bio-inspired control models outperform larger neural networks in small input-output spaces, and introduces a Parameter Impact metric.

Findings

Shallow MLPs and densely connected CPGs outperform deeper MLPs and Actor-Critic architectures.

Reinforcement learning's additional parameters do not improve performance.

Evolutionary strategies are more effective than reinforcement learning in this context.

Abstract

While Central Pattern Generators (CPGs) and Multi-Layer Perceptrons (MLP) are widely used paradigms in robot control, few systematic studies have been performed on the relative merits of large parameter spaces. In contexts where input and output spaces are small and performance is bounded, having more parameters to optimize may actively hinder the learning process instead of empowering it. To empirically measure this, we submit a given robot morphology, with limited proprioceptive capabilities, to controller optimization under two bio-inspired paradigms (CPGs and MLPs) with evolutionary- and reinforcement- trainer protocols. By varying parameter spaces across multiple reward functions, we observe that shallow MLPs and densely connected CPGs result in better performance when compared to deeper MLPs or Actor-Critic architectures. To account for the relationship between said performance and the number of parameters, we introduce a Parameter Impact metric which demonstrates that the additional parameters required by the reinforcement technique do not translate into better performance, thus favouring evolutionary strategies.