Last-Iterate Convergence of No-Regret Learning for Equilibria in Bargaining Games

TL;DR

This paper proves that a simple online learning algorithm, FTRL, can converge to approximate Nash equilibria in bargaining games like the ultimatum game, even though such games lack properties used in prior convergence proofs.

Contribution

It establishes last-iterate convergence of FTRL in bargaining games, including the ultimatum game, expanding understanding of equilibrium learning in complex economic settings.

Findings

FTRL converges to approximate NE in the ultimatum game.

Convergence occurs under broad initial conditions.

Experimental results support theoretical convergence in multi-round bargaining games.

Abstract

Bargaining games, where agents attempt to agree on how to split utility, are an important class of games used to study economic behavior, which motivates a study of online learning algorithms in these games. In this work, we tackle when no-regret learning algorithms converge to Nash equilibria in bargaining games. Recent results have shown that online algorithms related to Follow the Regularized Leader (FTRL) converge to Nash equilibria (NE) in the last iterate in a wide variety of games, including zero-sum games. However, bargaining games do not have the properties used previously to established convergence guarantees, even in the simplest case of the ultimatum game, which features a single take-it-or-leave-it offer. Nonetheless, we establish that FTRL (without the modifications necessary for zero-sum games) achieves last-iterate convergence to an approximate NE in the ultimatum game along with a bound on convergence time under mild assumptions. Further, we provide experimental results to demonstrate that convergence to NE, including NE with asymmetric payoffs, occurs under a broad range of initial conditions, both in the ultimatum game and in bargaining games with multiple rounds. This work demonstrates how complex economic behavior (e.g. learning to use threats and the existence of many possible equilibrium outcomes) can result from using a simple learning algorithm, and that FTRL can converge to equilibria in a more diverse set of games than previously known.