On characterization and existence of constrained correlated equilibria in Markov games

TL;DR

This paper investigates the existence and characterization of constrained correlated equilibria in Markov games, providing conditions under which such equilibria exist and exploring their properties for applications with safety and budget constraints.

Contribution

It introduces a new characterization of constrained correlated equilibria and establishes existence results under weakened Slater-type conditions, advancing the understanding of constrained strategic interactions.

Findings

Characterization of constrained correlated equilibria.

Existence of equilibria under weakened conditions.

Potential for efficient learning algorithms.

Abstract

Markov games with coupling constraints provide a natural framework to study constrained decision-making involving self-interested agents, where the feasibility of an individual agent's strategy depends on the joint strategies of the others. Such games arise in numerous real-world applications involving safety requirements and budget caps, for example, in environmental management, electricity markets, and transportation systems. While correlated equilibria have emerged as an important solution concept in unconstrained settings due to their computational tractability and amenability to learning, their constrained counterparts remain less explored. In this paper, we study constrained correlated equilibria-feasible policies where any unilateral modifications are either unprofitable or infeasible. We first characterize the constrained correlated equilibrium showing that different sets of modifications result in an equivalent notion, a result which may enable efficient learning algorithms. We then address existence conditions. In particular, we show that a strong Slater-type condition is necessary in games with playerwise coupling constraints, but can be significantly weakened when all players share common coupling constraints. Under this relaxed condition, we prove the existence of a constrained correlated equilibrium.