Learning to Coordinate Efficiently: A Model-based Approach

TL;DR

This paper introduces simple model-based algorithms for common-interest stochastic games that achieve polynomial convergence rates and guarantee convergence to the optimal value, improving upon existing reinforcement learning methods.

Contribution

The paper presents novel model-based algorithms that significantly enhance convergence speed and guarantee optimal solutions in common-interest stochastic games.

Findings

Achieve polynomial convergence rates

Guarantee convergence to the optimal value

Outperform existing reinforcement learning algorithms

Abstract

In common-interest stochastic games all players receive an identical payoff. Players participating in such games must learn to coordinate with each other in order to receive the highest-possible value. A number of reinforcement learning algorithms have been proposed for this problem, and some have been shown to converge to good solutions in the limit. In this paper we show that using very simple model-based algorithms, much better (i.e., polynomial) convergence rates can be attained. Moreover, our model-based algorithms are guaranteed to converge to the optimal value, unlike many of the existing algorithms.