Learning to Control Unknown Strongly Monotone Games

TL;DR

This paper introduces an online learning algorithm that adjusts control coefficients in strongly monotone games to steer the Nash equilibrium towards desired linear constraints without requiring knowledge of players' rewards or actions.

Contribution

It proposes a simple, privacy-preserving online method to control Nash equilibria in large-scale games by only using constraint violation feedback.

Findings

Algorithm converges with probability 1 to the set of constrained GNE.

Proven L2 convergence rate of approximately O(t^{-1/4}).

Applicable to large-scale networks with limited information.

Abstract

Consider a strongly monotone game where the players' utility functions include a reward function and a linear term for each dimension, with coefficients that are controlled by the manager. Gradient play converges to a unique Nash equilibrium (NE) that does not optimize the global objective. The global performance at NE can be improved by imposing linear constraints on the NE, also known as a generalized Nash equilibrium (GNE). We therefore want the manager to control the coefficients such that they impose the desired constraint on the NE. However, this requires knowing the players' rewards and action sets. Obtaining this game information is infeasible in a large-scale network and violates user privacy. To overcome this, we propose a simple algorithm that learns to shift the NE to meet the linear constraints by adjusting the controlled coefficients online. Our algorithm only requires the linear constraints violation as feedback and does not need to know the reward functions or the action sets. We prove that our algorithm converges with probability 1 to the set of GNE given by coupled linear constraints. We then prove an L2 convergence rate of near-$O(t^{-1/4})$.