Online and Offline Learning of Player Objectives from Partial Observations in Dynamic Games

TL;DR

This paper introduces a novel online method for learning players' objectives in dynamic games from noisy, partial observations, enabling real-time prediction and understanding of agent behaviors.

Contribution

It proposes a coupled estimation-inference approach that learns unknown objectives and unobserved states simultaneously within a receding horizon framework.

Findings

Accurately recovers players' preferences in simulated traffic scenarios.

Outperforms existing methods in noisy data conditions.

Enables real-time online learning and prediction.

Abstract

Robots deployed to the real world must be able to interact with other agents in their environment. Dynamic game theory provides a powerful mathematical framework for modeling scenarios in which agents have individual objectives and interactions evolve over time. However, a key limitation of such techniques is that they require a-priori knowledge of all players' objectives. In this work, we address this issue by proposing a novel method for learning players' objectives in continuous dynamic games from noise-corrupted, partial state observations. Our approach learns objectives by coupling the estimation of unknown cost parameters of each player with inference of unobserved states and inputs through Nash equilibrium constraints. By coupling past state estimates with future state predictions, our approach is amenable to simultaneous online learning and prediction in receding horizon fashion. We demonstrate our method in several simulated traffic scenarios in which we recover players' preferences for, e.g., desired travel speed and collision-avoidance behavior. Results show that our method reliably estimates game-theoretic models from noise-corrupted data that closely matches ground-truth objectives, consistently outperforming state-of-the-art approaches.