Coordinated Online Learning for Multi-Agent Systems with Coupled Constraints and Perturbed Utility Observations

TL;DR

This paper introduces a decentralized online learning method for multi-agent systems with coupled resource constraints, ensuring convergence to equilibrium despite noisy utility feedback, applicable to large-scale resource allocation problems.

Contribution

A novel decentralized resource pricing algorithm that guarantees convergence to a generalized Nash equilibrium under noisy feedback conditions.

Findings

Almost sure convergence to generalized Nash equilibrium

Resource constraints are asymptotically satisfied

Finite-time bounds on resource constraint violations

Abstract

Competitive non-cooperative online decision-making agents whose actions increase congestion of scarce resources constitute a model for widespread modern large-scale applications. To ensure sustainable resource behavior, we introduce a novel method to steer the agents toward a stable population state, fulfilling the given coupled resource constraints. The proposed method is a decentralized resource pricing method based on the resource loads resulting from the augmentation of the game's Lagrangian. Assuming that the online learning agents have only noisy first-order utility feedback, we show that for a polynomially decaying agents' step size/learning rate, the population's dynamic will almost surely converge to generalized Nash equilibrium. A particular consequence of the latter is the fulfillment of resource constraints in the asymptotic limit. Moreover, we investigate the finite-time quality of the proposed algorithm by giving a nonasymptotic time decaying bound for the expected amount of resource constraint violation.