A Strategic Learning Algorithm for State-based Games

TL;DR

This paper introduces a heuristic, uncoupled learning algorithm for state-based games that guarantees convergence to equilibrium under certain conditions, using finite memory, inertia, and randomness.

Contribution

It proposes a novel two-memory better reply with inertia dynamics algorithm and analyzes the existence of universal learning algorithms in state-based games.

Findings

Algorithm converges to recurrent state equilibria under specified conditions.

No universal learning algorithm guarantees convergence in all state-based games.

The approach combines global and local searches with finite memory and randomness.

Abstract

Learning algorithm design for state-based games is investigated. A heuristic uncoupled learning algorithm, which is a two memory better reply with inertia dynamics, is proposed. Under certain reasonable conditions it is proved that for any initial state, if all agents in the state-based game follow the proposed learning algorithm, the action state pair converges almost surely to an action invariant set of recurrent state equilibria. The design relies on global and local searches with finite memory, inertia, and randomness. Finally, existence of time-efficient universal learning algorithm is studied. A class of state-based games is presented to show that there is no universal learning algorithm converging to a recurrent state equilibrium.