Distributionally Robust Inverse Reinforcement Learning for Identifying Multi-Agent Coordinated Sensing

TL;DR

This paper introduces a distributionally robust inverse reinforcement learning method to accurately infer utility functions of multi-agent sensing systems under uncertainty, demonstrated through numerical experiments on a cognitive radar network.

Contribution

It develops a novel minimax IRL algorithm using Wasserstein ambiguity sets, with proven reformulation and a practical solution approach.

Findings

Effective reconstruction of utility functions in noisy environments

Robust IRL outperforms non-robust methods in accuracy

Applicable to multi-agent sensing systems like cognitive radar

Abstract

We derive a minimax distributionally robust inverse reinforcement learning (IRL) algorithm to reconstruct the utility functions of a multi-agent sensing system. Specifically, we construct utility estimators which minimize the worst-case prediction error over a Wasserstein ambiguity set centered at noisy signal observations. We prove the equivalence between this robust estimation and a semi-infinite optimization reformulation, and we propose a consistent algorithm to compute solutions. We illustrate the efficacy of this robust IRL scheme in numerical studies to reconstruct the utility functions of a cognitive radar network from observed tracking signals.