Efficient Iterative Linear-Quadratic Approximations for Nonlinear Multi-Player General-Sum Differential Games

TL;DR

This paper introduces an efficient iterative algorithm for solving nonlinear multi-player differential games, enabling real-time decision-making in complex multi-agent robotic scenarios by approximating the game as a series of linear-quadratic problems.

Contribution

The paper proposes a novel iterative linear-quadratic game-solving algorithm inspired by ILQR, tailored for nonlinear multi-player differential games, with demonstrated real-time performance and convergence.

Findings

Algorithm converges reliably across various initial conditions.

Achieves real-time performance with convergence times under 50 ms.

Produces strategies exhibiting complex interactive behaviors.

Abstract

Many problems in robotics involve multiple decision making agents. To operate efficiently in such settings, a robot must reason about the impact of its decisions on the behavior of other agents. Differential games offer an expressive theoretical framework for formulating these types of multi-agent problems. Unfortunately, most numerical solution techniques scale poorly with state dimension and are rarely used in real-time applications. For this reason, it is common to predict the future decisions of other agents and solve the resulting decoupled, i.e., single-agent, optimal control problem. This decoupling neglects the underlying interactive nature of the problem; however, efficient solution techniques do exist for broad classes of optimal control problems. We take inspiration from one such technique, the iterative linear-quadratic regulator (ILQR), which solves repeated approximations with linear dynamics and quadratic costs. Similarly, our proposed algorithm solves repeated linear-quadratic games. We experimentally benchmark our algorithm in several examples with a variety of initial conditions and show that the resulting strategies exhibit complex interactive behavior. Our results indicate that our algorithm converges reliably and runs in real-time. In a three-player, 14-state simulated intersection problem, our algorithm initially converges in < 0.25s. Receding horizon invocations converge in < 50 ms in a hardware collision-avoidance test.