Reinforcement learning with restrictions on the action set

TL;DR

This paper introduces an adaptive learning method for repeated two-player games with action restrictions, showing convergence to Nash equilibria in specific game classes despite limited information and action sets.

Contribution

It develops a novel learning procedure where players only observe their payoffs and have restricted actions, proving convergence in certain game types.

Findings

Empirical distributions converge to Nash equilibria in zero-sum and potential games.

Convergence also occurs in games where one player has only two actions.

The method works despite players not knowing their payoff functions or observing the opponent.

Abstract

Consider a 2-player normal-form game repeated over time. We introduce an adaptive learning procedure, where the players only observe their own realized payoff at each stage. We assume that agents do not know their own payoff function, and have no information on the other player. Furthermore, we assume that they have restrictions on their own action set such that, at each stage, their choice is limited to a subset of their action set. We prove that the empirical distributions of play converge to the set of Nash equilibria for zero-sum and potential games, and games where one player has two actions.