The dynamical regime and its importance for evolvability, task performance and generalization

TL;DR

This study investigates how the dynamical regime of neural network-controlled agents affects their evolvability, performance, and adaptability, revealing that subcritical regimes often outperform critical ones in simple tasks, but harder tasks favor near-criticality.

Contribution

It demonstrates that evolving agents tend to become subcritical in simple tasks and highlights the importance of initial proximity to criticality for optimal adaptability and performance.

Findings

Subcritical regimes often outperform critical regimes in simple tasks.

Agents near criticality maintain fitness better under environmental changes.

Optimal dynamical regime depends on task difficulty, with harder tasks favoring near-criticality.

Abstract

It has long been hypothesized that operating close to the critical state is beneficial for natural and artificial systems. We test this hypothesis by evolving foraging agents controlled by neural networks that can change the system's dynamical regime throughout evolution. Surprisingly, we find that all populations, regardless of their initial regime, evolve to be subcritical in simple tasks and even strongly subcritical populations can reach comparable performance. We hypothesize that the moderately subcritical regime combines the benefits of generalizability and adaptability brought by closeness to criticality with the stability of the dynamics characteristic for subcritical systems. By a resilience analysis, we find that initially critical agents maintain their fitness level even under environmental changes and degrade slowly with increasing perturbation strength. On the other hand, subcritical agents originally evolved to the same fitness, were often rendered utterly inadequate and degraded faster. We conclude that although the subcritical regime is preferable for a simple task, the optimal deviation from criticality depends on the task difficulty: for harder tasks, agents evolve closer to criticality. Furthermore, subcritical populations cannot find the path to decrease their distance to criticality. In summary, our study suggests that initializing models near criticality is important to find an optimal and flexible solution.