Multi-agent learning using Fictitious Play and Extended Kalman Filter

TL;DR

This paper introduces a novel fictitious play variant using Extended Kalman filters for strategy prediction, demonstrating improved convergence and performance in multi-agent decentralized optimization tasks.

Contribution

The paper proposes a new fictitious play algorithm with Kalman filter-based opponent strategy prediction, enhancing convergence in potential and 2x2 games.

Findings

Converges to pure Nash equilibrium in specific game settings.

Outperforms classic fictitious play in tested scenarios.

Improves decentralized optimization performance.

Abstract

Decentralised optimisation tasks are important components of multi-agent systems. These tasks can be interpreted as n-player potential games: therefore game-theoretic learning algorithms can be used to solve decentralised optimisation tasks. Fictitious play is the canonical example of these algorithms. Nevertheless fictitious play implicitly assumes that players have stationary strategies. We present a novel variant of fictitious play where players predict their opponents' strategies using Extended Kalman filters and use their predictions to update their strategies. We show that in 2 by 2 games with at least one pure Nash equilibrium and in potential games where players have two available actions, the proposed algorithm converges to the pure Nash equilibrium. The performance of the proposed algorithm was empirically tested, in two strategic form games and an ad-hoc sensor network surveillance problem. The proposed algorithm performs better than the classic fictitious play algorithm in these games and therefore improves the performance of game-theoretical learning in decentralised optimisation.