Nash Convergence of Gradient Dynamics in Iterated General-Sum Games

TL;DR

This paper analyzes how gradient ascent agents in two-player, two-action iterated general-sum games tend to converge to Nash equilibria, either through strategies or average payoffs, revealing a surprising convergence behavior.

Contribution

It provides a novel theoretical analysis of gradient dynamics in general-sum games, showing convergence to Nash equilibria in strategy or payoff.

Findings

Agents converge to Nash equilibrium strategies or payoffs.

Average payoffs tend to Nash equilibrium payoffs even if strategies do not.

The convergence behavior is surprisingly robust in simple iterated games.

Abstract

Multi-agent games are becoming an increasing prevalent formalism for the study of electronic commerce and auctions. The speed at which transactions can take place and the growing complexity of electronic marketplaces makes the study of computationally simple agents an appealing direction. In this work, we analyze the behavior of agents that incrementally adapt their strategy through gradient ascent on expected payoff, in the simple setting of two-player, two-action, iterated general-sum games, and present a surprising result. We show that either the agents will converge to Nash equilibrium, or if the strategies themselves do not converge, then their average payoffs will nevertheless converge to the payoffs of a Nash equilibrium.