Adversarial Coevolutionary Illumination with Generational Adversarial MAP-Elites

TL;DR

The paper introduces GAME, a coevolutionary QD algorithm that evolves both sides of adversarial problems using a vision embedding model, outperforming one-sided baselines across multiple domains.

Contribution

It presents a novel generational adversarial MAP-Elites algorithm that coevolves adversaries without domain-specific descriptors, validated on diverse adversarial tasks.

Findings

GAME finds better solutions than one-sided QD baselines.

The VEM effectively operates on video data in multiple domains.

Evolutionary phenomena like arms races and neutral mutations are observed.

Abstract

Quality-Diversity (QD) algorithms seek to discover diverse, high-performing solutions across a behavior space, in contrast to conventional optimization methods that target a single optimum. Adversarial problems present unique challenges for QD approaches, as the competing nature of opposing sides creates interdependencies that complicate the evolution process. Existing QD methods applied to such scenarios typically fix one side, constraining the open-endedness. We present Generational Adversarial MAP-Elites (GAME), a coevolutionary QD algorithm that evolves both sides by alternating which side is evolved at each generation. By integrating a vision embedding model (VEM), our approach eliminates the need for domain-specific behavior descriptors and instead operates on video. We validate GAME across three distinct adversarial domains: a multi-agent battle game, a soft-robot wrestling environment, and a deck building game. We validate that all its components are necessary, that the VEM is effective in two different domains, and that GAME finds better solutions than one-sided QD baselines. Our experiments reveal several evolutionary phenomena, including arms race-like dynamics, enhanced novelty through generational extinction, and the preservation of neutral mutations as crucial stepping stones toward the highest performance. While GAME successfully illuminates all three adversarial problems, its capacity for truly open-ended discovery remains constrained by the nature of the search spaces used in this paper. These findings show GAME's broad applicability and highlight opportunities for future research into open-ended adversarial coevolution. Code and videos are available at: https://github.com/Timothee-ANNE/GAME