Efficiently Solving Mixed-Hierarchy Games with Quasi-Policy Approximations

TL;DR

This paper introduces a quasi-policy approximation method and an inexact Newton algorithm to efficiently solve complex multi-robot mixed-hierarchy games, enabling real-time convergence in practical scenarios.

Contribution

It develops a novel quasi-policy approximation and an efficient Newton-based solver for mixed-hierarchy multi-robot games with proven local convergence.

Findings

Achieves real-time convergence in hardware and simulation experiments.

Demonstrates effectiveness for complex mixed-hierarchy information structures.

Provides a Julia library for practical implementation.

Abstract

Multi-robot coordination often exhibits hierarchical structure, with some robots' decisions depending on the planned behaviors of others. While game theory provides a principled framework for such interactions, existing solvers struggle to handle mixed information structures that combine simultaneous (Nash) and hierarchical (Stackelberg) decision-making. We study N-robot forest-structured mixed-hierarchy games, in which each robot acts as a Stackelberg leader over its subtree while robots in different branches interact via Nash equilibria. We derive the Karush-Kuhn-Tucker (KKT) first-order optimality conditions for this class of games and show that they involve increasingly high-order derivatives of robots' best-response policies as the hierarchy depth grows, rendering a direct solution intractable. To overcome this challenge, we introduce a quasi-policy approximation that removes higher-order policy derivatives and develop an inexact Newton method for efficiently solving the resulting approximated KKT systems. We prove local exponential convergence of the proposed algorithm for games with non-quadratic objectives and nonlinear constraints. The approach is implemented in a highly optimized Julia library (MixedHierarchyGames.jl) and evaluated in hardware and simulated multi-agent experiments, demonstrating real-time convergence for complex mixed-hierarchy information structures.