Learning Pure Nash Equilibrium in Smart Charging Games

TL;DR

This paper extends reinforcement learning convergence results to finite congestion games with non-separable costs, including smart charging scenarios, by establishing an ordinal potential function that guarantees pure Nash equilibrium convergence.

Contribution

It introduces a new class of congestion games with non-separable costs, proves the existence of an ordinal potential function, and demonstrates convergence of RL algorithms in this setting.

Findings

Finite congestion games with non-separable costs have an ordinal potential function.

Reinforcement learning algorithms converge to pure Nash equilibria in these games.

A smart charging game example illustrates the convergence results.

Abstract

Reinforcement Learning Algorithms (RLA) are useful machine learning tools to understand how decision makers react to signals. It is known that RLA converge towards the pure Nash Equilibria (NE) of finite congestion games and more generally, finite potential games. For finite congestion games, only separable cost functions are considered. However, non-separable costs, which depend on the choices of all players instead of only those choosing the same resource, may be relevant in some circumstances, like in smart charging games. In this paper, finite congestion games with non-separable costs are shown to have an ordinal potential function, leading to the existence of an action-dependent continuous potential function. The convergence of a synchronous RLA towards the pure NE is then extended to this more general class of congestion games. Finally, a smart charging game is designed for illustrating convergence of such learning algorithms.