Inverse Reinforcement Learning for Strategy Identification

TL;DR

This paper introduces a framework using inverse reinforcement learning to identify opponent strategies in adversarial environments, demonstrated through gaming data, with recovered rewards visualized, clustered, and classified.

Contribution

It presents a novel IRL-based framework for strategy identification in adversarial settings, validated on gaming data with multiple analysis techniques.

Findings

Recovered rewards can be effectively visualized.

Unsupervised clustering groups similar strategies.

Supervised learning classifies strategies accurately.

Abstract

In adversarial environments, one side could gain an advantage by identifying the opponent's strategy. For example, in combat games, if an opponents strategy is identified as overly aggressive, one could lay a trap that exploits the opponent's aggressive nature. However, an opponent's strategy is not always apparent and may need to be estimated from observations of their actions. This paper proposes to use inverse reinforcement learning (IRL) to identify strategies in adversarial environments. Specifically, the contributions of this work are 1) the demonstration of this concept on gaming combat data generated from three pre-defined strategies and 2) the framework for using IRL to achieve strategy identification. The numerical experiments demonstrate that the recovered rewards can be identified using a variety of techniques. In this paper, the recovered reward are visually displayed, clustered using unsupervised learning, and classified using a supervised learner.