Linear-Quadratic Discrete-Time Dynamic Games with Unknown Dynamics

TL;DR

This paper develops a data-driven framework for solving finite and infinite-horizon linear-quadratic discrete-time dynamic games with unknown dynamics, providing conditions for feedback Nash equilibria and algorithms for their computation.

Contribution

It introduces necessary and sufficient conditions for FNE existence using offline data and proposes algorithms for computing equilibria without known system models.

Findings

FNEs of unknown and known dynamics games are equivalent.

An invertibility condition simplifies FNE computation.

Finite-horizon strategies converge to infinite-horizon solutions.

Abstract

Considering linear-quadratic discrete-time games with unknown input/output/state (i/o/s) dynamics and state, we provide necessary and sufficient conditions for the existence and uniqueness of feedback Nash equilibria (FNE) in the finite-horizon game, based entirely on offline input/output data. We prove that the finite-horizon unknown-dynamics game and its corresponding known-dynamics game have the same FNEs, and provide detailed relationships between their respective FNE matrices. To simplify the computation of FNEs, we provide an invertibility condition and a corresponding algorithm that computes one FNE by solving a finite number of linear equation systems using offline data. For the infinite-horizon unknown-dynamics game, limited offline data restricts players to computing optimal strategies only over a finite horizon. We prove that the finite-horizon strategy ``watching $T$ steps into the future and moving one step now,'' which is commonly used in classical optimal control, exhibits convergence in both the FNE matrices and the total costs in the infinite-horizon unknown-dynamics game, and further provide an analysis of the convergence rate of the total cost. The corresponding algorithm for the infinite-horizon game is proposed and its efficacy is demonstrated through a non-scalar numerical example.