Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization

TL;DR

This paper introduces a unified framework for recovering reward functions in competitive games using inverse game theory with entropy regularization, supported by theoretical guarantees and practical algorithms.

Contribution

It establishes reward identifiability via QRE, proposes a novel algorithm for reward learning from observed actions, and provides theoretical and empirical validation.

Findings

Reward functions are identifiable under linear assumptions using QRE.

The proposed algorithm works in static and dynamic settings.

Numerical studies demonstrate the framework's effectiveness.

Abstract

Estimating the unknown reward functions driving agents' behaviors is of central interest in inverse reinforcement learning and game theory. To tackle this problem, we develop a unified framework for reward function recovery in two-player zero-sum matrix games and Markov games with entropy regularization, where we aim to reconstruct the underlying reward functions given observed players' strategies and actions. This task is challenging due to the inherent ambiguity of inverse problems, the non-uniqueness of feasible rewards, and limited observational data coverage. To address these challenges, we establish the reward function's identifiability using the quantal response equilibrium (QRE) under linear assumptions. Building upon this theoretical foundation, we propose a novel algorithm to learn reward functions from observed actions. Our algorithm works in both static and dynamic settings and is adaptable to incorporate different methods, such as Maximum Likelihood Estimation (MLE). We provide strong theoretical guarantees for the reliability and sample efficiency of our algorithm. Further, we conduct extensive numerical studies to demonstrate the practical effectiveness of the proposed framework, offering new insights into decision-making in competitive environments.