From Weak Learning to Strong Learning in Fictitious Play Type Algorithms

TL;DR

This paper analyzes Fictitious Play and related algorithms, identifying their limitations in convergence, and proposes modifications to ensure strategies converge to equilibrium, with proven theoretical guarantees.

Contribution

It introduces a method to modify FP-type algorithms to achieve strong learning, ensuring convergence of strategies to equilibrium.

Findings

Proves convergence of the modified algorithms to equilibrium strategies.

Identifies the discontinuity issue as a key obstacle in FP convergence.

Provides theoretical guarantees for the proposed modifications.

Abstract

The paper studies the highly prototypical Fictitious Play (FP) algorithm, as well as a broad class of learning processes based on best-response dynamics, that we refer to as FP-type algorithms. A well-known shortcoming of FP is that, while players may learn an equilibrium strategy in some abstract sense, there are no guarantees that the period-by-period strategies generated by the algorithm actually converge to equilibrium themselves. This issue is fundamentally related to the discontinuous nature of the best response correspondence and is inherited by many FP-type algorithms. Not only does it cause problems in the interpretation of such algorithms as a mechanism for economic and social learning, but it also greatly diminishes the practical value of these algorithms for use in distributed control. We refer to forms of learning in which players learn equilibria in some abstract sense only (to be defined more precisely in the paper) as weak learning, and we refer to forms of learning where players' period-by-period strategies converge to equilibrium as strong learning. An approach is presented for modifying an FP-type algorithm that achieves weak learning in order to construct a variant that achieves strong learning. Theoretical convergence results are proved.