On convergence rates of game theoretic reinforcement learning algorithms

TL;DR

This paper introduces a reinforcement learning algorithm for multi-player games with limited information, demonstrating almost sure convergence to optimal action profiles and quantifying the convergence rate, validated through smart grid and cybersecurity case studies.

Contribution

The paper proposes a novel reinforcement learning algorithm for multi-player games with partial information, providing convergence guarantees and rate analysis.

Findings

Algorithm converges to maximal stochastic potential profiles with probability one.

Convergence rate of the algorithm is explicitly quantified.

Performance validated through case studies in smart grid and cybersecurity.

Abstract

This paper investigates a class of multi-player discrete games where each player aims to maximize its own utility function. Each player does not know the other players' action sets, their deployed actions or the structures of its own or the others' utility functions. Instead, each player only knows its own deployed actions and its received utility values in recent history. We propose a reinforcement learning algorithm which converges to the set of action profiles which have maximal stochastic potential with probability one. Furthermore, the convergence rate of the proposed algorithm is quantified. The algorithm performance is verified using two case studies in the smart grid and cybersecurity.