Pick Your Battles: Interaction Graphs as Population-Level Objectives for Strategic Diversity

TL;DR

This paper introduces interaction graphs to structure agent interactions during training, promoting strategic diversity and improving multi-agent performance across various games.

Contribution

It proposes a novel method using interaction graphs to foster diversity in agent populations, enhancing training effectiveness and strategic coverage.

Findings

Interaction graphs influence training trajectories and diversity.

Structured interactions improve multi-agent performance.

Diversity benefits are demonstrated across multiple games.

Abstract

Strategic diversity is often essential in games: in multi-player games, for example, evaluating a player against a diverse set of strategies will yield a more accurate estimate of its performance. Furthermore, in games with non-transitivities diversity allows a player to cover several winning strategies. However, despite the significance of strategic diversity, training agents that exhibit diverse behaviour remains a challenge. In this paper we study how to construct diverse populations of agents by carefully structuring how individuals within a population interact. Our approach is based on interaction graphs, which control the flow of information between agents during training and can encourage agents to specialise on different strategies, leading to improved overall performance. We provide evidence for the importance of diversity in multi-agent training and analyse the effect of applying different interaction graphs on the training trajectories, diversity and performance of populations in a range of games. This is an extended version of the long abstract published at AAMAS.