Disturbance Decoupling for Gradient-based Multi-Agent Learning with Quadratic Costs

TL;DR

This paper investigates how certain players in multi-agent gradient-based learning can be unaffected by disturbances, providing algebraic and graph-theoretic conditions for quadratic cost games.

Contribution

It establishes necessary and sufficient conditions for disturbance decoupling in quadratic cost multi-agent games, linking algebraic properties and graph structures.

Findings

Disturbance decoupling depends on algebraic and graph-theoretic conditions.

In LQ games, decoupling constrains controllable and unobservable subspaces.

In bilinear games, decoupling imposes constraints on payoff matrices.

Abstract

Motivated by applications of multi-agent learning in noisy environments, this paper studies the robustness of gradient-based learning dynamics with respect to disturbances. While disturbances injected along a coordinate corresponding to any individual player's actions can always affect the overall learning dynamics, a subset of players can be disturbance decoupled---i.e., such players' actions are completely unaffected by the injected disturbance. We provide necessary and sufficient conditions to guarantee this property for games with quadratic cost functions, which encompass quadratic one-shot continuous games, finite-horizon linear quadratic (LQ) dynamic games, and bilinear games. Specifically, disturbance decoupling is characterized by both algebraic and graph-theoretic conditions on the learning dynamics, the latter is obtained by constructing a game graph based on gradients of players' costs. For LQ games, we show that disturbance decoupling imposes constraints on the controllable and unobservable subspaces of players. For two player bilinear games, we show that disturbance decoupling within a player's action coordinates imposes constraints on the payoff matrices. Illustrative numerical examples are provided.