Evaluation and Learning in Two-Player Symmetric Games via Best and Better Responses

TL;DR

This paper introduces new metrics and learning algorithms for strategy evaluation and selection in two-player symmetric games, demonstrating their effectiveness through theoretical analysis and simulations.

Contribution

It presents two novel digraph-based metrics for strategy evaluation and two variants of self-play algorithms tailored for symmetric games, linking theoretical properties with practical learning.

Findings

Strategies learned by the algorithms are preferred under the proposed metrics.

The metrics effectively rank strategies and identify preferred ones.

Simulations validate the theoretical results and demonstrate the approach.

Abstract

Artificial intelligence and robotic competitions are accompanied by a class of game paradigms in which each player privately commits a strategy to a game system which simulates the game using the collected joint strategy and then returns payoffs to players. This paper considers the strategy commitment for two-player symmetric games in which the players' strategy spaces are identical and their payoffs are symmetric. First, we introduce two digraph-based metrics at a meta-level for strategy evaluation in two-agent reinforcement learning, grounded on sink equilibrium. The metrics rank the strategies of a single player and determine the set of strategies which are preferred for the private commitment. Then, in order to find the preferred strategies under the metrics, we propose two variants of the classical learning algorithm self-play, called strictly best-response and weakly better-response self-plays. By modeling learning processes as walks over joint-strategy response digraphs, we prove that the learnt strategies by two variants are preferred under two metrics, respectively. The preferred strategies under both two metrics are identified and adjacency matrices induced by one metric and one variant are connected. Finally, simulations are provided to illustrate the results.