Forward-Backward Dynamic Programming for LQG Dynamic Games with Partial and Asymmetric Information

TL;DR

This paper introduces a novel iterative forward-backward algorithm for solving two-player zero-sum stochastic dynamic games with partial and asymmetric information within an LQG framework, addressing belief representation and strategy computation.

Contribution

It develops a new computational approach combining forward-backward and value iteration algorithms for belief and strategy computation in asymmetric information games.

Findings

Algorithms effectively compute equilibrium strategies and belief states.

Numerical experiments demonstrate the algorithms' practical effectiveness.

Open-source implementation is provided for reproducibility.

Abstract

We formulate and study a class of two-player zero-sum stochastic dynamic games with partial and asymmetric information. Information asymmetry introduces fundamental challenges involving \emph{belief representation} and \emph{theory of mind} issues, where agents must impute belief states and estimates of other agents to inform their own strategy. To avoid an infinite regress of higher-order beliefs amongst agents and obtain computationally implementable results, we focus on a linear quadratic Gaussian (LQG) model and consider strategies with limited internal state dimension. We present a novel iterative forward-backward algorithm to jointly compute belief states and equilibrium strategies and value functions for a finite-horizon problem. We also present a value iteration-like algorithm to jointly compute stationary belief states and equilibrium strategies for an average-cost infinite-horizon problem. An open-source implementation of the algorithms is provided, and we demonstrate the effectiveness of the proposed algorithms in numerical experiments.