Adaptive Forgetting Factor Fictitious Play

TL;DR

This paper introduces an adaptive fictitious play algorithm that dynamically updates opponent strategy estimates, improving convergence rates in decentralized optimization scenarios compared to traditional methods.

Contribution

It proposes a novel adaptive fictitious play method that incorporates online streaming data techniques to better model non-stationary opponent strategies.

Findings

Faster or comparable convergence rates to existing fictitious play variants.

Effective in strategic form, vehicle target assignment, and disaster management problems.

Enhances game-theoretic learning performance in decentralized optimization.

Abstract

It is now well known that decentralised optimisation can be formulated as a potential game, and game-theoretical learning algorithms can be used to find an optimum. One of the most common learning techniques in game theory is fictitious play. However fictitious play is founded on an implicit assumption that opponents' strategies are stationary. We present a novel variation of fictitious play that allows the use of a more realistic model of opponent strategy. It uses a heuristic approach, from the online streaming data literature, to adaptively update the weights assigned to recently observed actions. We compare the results of the proposed algorithm with those of stochastic and geometric fictitious play in a simple strategic form game, a vehicle target assignment game and a disaster management problem. In all the tests the rate of convergence of the proposed algorithm was similar or better than the variations of fictitious play we compared it with. The new algorithm therefore improves the performance of game-theoretical learning in decentralised optimisation.