Best Response Convergence for Zero-sum Stochastic Dynamic Games with Partial and Asymmetric Information

TL;DR

This paper investigates the convergence of best response dynamics in zero-sum stochastic linear quadratic dynamic games with partial and asymmetric information, showing that strategies with limited internal states can approximate Nash equilibria effectively.

Contribution

It derives explicit best response formulas for players and demonstrates that higher-order belief states rapidly become intractable, leading to convergence with low-dimensional strategies.

Findings

Game value converges after few iterations in numerical experiments.

Eigenvalues of controllability and observability Gramians decay rapidly in higher-order beliefs.

Low-order belief dynamics closely approximate high-order strategies with bounded error.

Abstract

We analyze best response dynamics for finding a Nash equilibrium of an infinite horizon zero-sum stochastic linear quadratic dynamic game (LQDG) with partial and asymmetric information. We derive explicit expressions for each player's best response within the class of pure linear dynamic output feedback control strategies where the internal state dimension of each control strategy is an integer multiple of the system state dimension. With each best response, the players form increasingly higher-order belief states, leading to infinite-dimensional internal states. However, we observe in extensive numerical experiments that the game's value converges after just a few iterations, suggesting that strategies associated with increasingly higher-order belief states eventually provide no benefit. To help explain this convergence, our numerical analysis reveals rapid decay of the controllability and observability Gramian eigenvalues and Hankel singular values in higher-order belief dynamics, indicating that the higher-order belief dynamics become increasingly difficult for both players to control and observe. Consequently, the higher-order belief dynamics can be closely approximated by low-order belief dynamics with bounded error, and thus feedback strategies with limited internal state dimension can closely approximate a Nash equilibrium.