Transfer Dynamics in Emergent Evolutionary Curricula

TL;DR

This paper investigates how transfer of policies between species in an open-ended neuroevolution system fosters diversity and success, emphasizing the crucial role of rare inter-species transfers in emergent evolution.

Contribution

It provides a detailed analysis of transfer dynamics in open-ended evolutionary curricula, highlighting the importance of inter-species transfer for system success.

Findings

Inter-species transfer is rare but essential for success.

Minimal criteria promote level diversity.

Phylogenetic analysis reveals transfer patterns.

Abstract

PINSKY is a system for open-ended learning through neuroevolution in game-based domains. It builds on the Paired Open-Ended Trailblazer (POET) system, which originally explored learning and environment generation for bipedal walkers, and adapts it to games in the General Video Game AI (GVGAI) system. Previous work showed that by co-evolving levels and neural network policies, levels could be found for which successful policies could not be created via optimization alone. Studied in the realm of Artificial Life as a potentially open-ended alternative to gradient-based fitness, minimal criteria (MC)-based selection helps foster diversity in evolutionary populations. The main question addressed by this paper is how the open-ended learning actually works, focusing in particular on the role of transfer of policies from one evolutionary branch ("species") to another. We analyze the dynamics of the system through creating phylogenetic trees, analyzing evolutionary trajectories of policies, and temporally breaking down transfers according to species type. Furthermore, we analyze the impact of the minimal criterion on generated level diversity and inter-species transfer. The most insightful finding is that inter-species transfer, while rare, is crucial to the system's success.