Approximate Feedback Nash Equilibria with Sparse Inter-Agent Dependencies

TL;DR

This paper introduces a regularized dynamic programming method to compute sparse feedback Nash equilibrium strategies in multi-agent games, reducing reliance on full state information and improving robustness under noisy observations.

Contribution

It develops a convex group Lasso-based approach for approximating Nash equilibria with sparse policies and extends it to non-linear games with an iterative algorithm.

Findings

Sparse policies effectively approximate Nash equilibria.

Regularized policies outperform standard policies under noisy observations.

Simulation shows up to 77% cost reduction with regularized policies.

Abstract

Feedback Nash equilibrium strategies in multi-agent dynamic games require availability of all players' state information to compute control actions. However, in real-world scenarios, sensing and communication limitations between agents make full state feedback expensive or impractical, and such strategies can become fragile when state information from other agents is inaccurate. To this end, we propose a regularized dynamic programming approach for finding sparse feedback policies that selectively depend on the states of a subset of agents in dynamic games. The proposed approach solves convex adaptive group Lasso problems to compute sparse policies approximating Nash equilibrium solutions. We prove the regularized solutions' asymptotic convergence to a neighborhood of Nash equilibrium policies in linear-quadratic (LQ) games. Further, we extend the proposed approach to general non-LQ games via an iterative algorithm. Simulation results in multi-robot interaction scenarios show that the proposed approach effectively computes feedback policies with varying sparsity levels. When agents have noisy observations of other agents' states, simulation results indicate that the proposed regularized policies consistently achieve lower costs than standard Nash equilibrium policies by up to 77% for all interacting agents whose costs are coupled with other agents' states.