Stochastic Dynamic Games in Belief Space

TL;DR

This paper introduces a novel approach using local iterative dynamic programming in Gaussian belief space to solve stochastic non-cooperative dynamic games in continuous POMDPs, enabling effective multi-agent information gathering under uncertainty.

Contribution

It develops a game-theoretic belief-space planning method using quadratic games and iterative dynamic programming, providing scalable solutions for multi-agent stochastic games.

Findings

Agents with the proposed method win 44% more races than without game theory.

The approach yields linear feedback policies for robots and predicted policies for others.

Runtime is polynomial in the number of agents and linear in the planning horizon.

Abstract

Information gathering while interacting with other agents under sensing and motion uncertainty is critical in domains such as driving, service robots, racing, or surveillance. The interests of agents may be at odds with others, resulting in a stochastic non-cooperative dynamic game. Agents must predict others' future actions without communication, incorporate their actions into these predictions, account for uncertainty and noise in information gathering, and consider what information their actions reveal. Our solution uses local iterative dynamic programming in Gaussian belief space to solve a game-theoretic continuous POMDP. Solving a quadratic game in the backward pass of a game-theoretic belief-space variant of iLQG achieves a runtime polynomial in the number of agents and linear in the planning horizon. Our algorithm yields linear feedback policies for our robot, and predicted feedback policies for other agents. We present three applications: active surveillance, guiding eyes for a blind agent, and autonomous racing. Agents with game-theoretic belief-space planning win 44% more races than without game theory and 34% more than without belief-space planning.