Evolutionary and Coevolutionary Multi-Agent Design Choices and Dynamics

TL;DR

This paper compares different controller representations and evolutionary algorithms, including a novel LLM-supported mutation, in coevolving cyber agent teams to evaluate their impact on performance and dynamics.

Contribution

It introduces a grammar-based controller representation with a new LLM-supported mutation operator and assesses its effectiveness in coevolutionary cyber agent scenarios.

Findings

Grammar-based controllers achieve top team performance.

Coevolution reduces performance variability but causes fluctuations.

Single-side optimization yields higher and more sustained performance.

Abstract

We investigate two representation alternatives for the controllers of teams of cyber agents. We combine these controller representations with different evolutionary algorithms, one of which introduces a novel LLM-supported mutation operator. Using a cyber security scenario, we evaluate agent learning when one side is trained to compete against a side that does not evolve and when two sides coevolve with each other. This allows us to quantify the relative merits and tradeoffs of representation and algorithm combinations in terms of team performance. Our versions of grammatical evolution algorithms using grammars that allow a controller to be expressed in code-like logic can achieve the best team performance. The scenario also allows us to compare the performance impact and dynamics of coevolution versus evolution under different combinations. Across the algorithms and representations, we observe that coevolution reduces the performance highs and lows of both sides while it induces fluctuations on both sides. In contrast, when only one-side is optimized, performance peaks are higher and is more sustained than when both sides are optimized with coevolution.