Diversity-based Deep Reinforcement Learning Towards Multidimensional Difficulty for Fighting Game AI

TL;DR

This paper proposes a diversity-driven deep reinforcement learning method to create multi-strategy AI agents for fighting games, making AI opponents more human-like by exhibiting varied tactics at each difficulty level.

Contribution

It introduces a novel diversity-based training approach for deep reinforcement learning to generate multiple strategies at similar difficulty levels, surpassing traditional reward-based methods.

Findings

Outperforms baseline in diversity of strategies

Achieves higher performance in generating human-like AI

Demonstrates effectiveness in fighting game AI

Abstract

In fighting games, individual players of the same skill level often exhibit distinct strategies from one another through their gameplay. Despite this, the majority of AI agents for fighting games have only a single strategy for each "level" of difficulty. To make AI opponents more human-like, we'd ideally like to see multiple different strategies at each level of difficulty, a concept we refer to as "multidimensional" difficulty. In this paper, we introduce a diversity-based deep reinforcement learning approach for generating a set of agents of similar difficulty that utilize diverse strategies. We find this approach outperforms a baseline trained with specialized, human-authored reward functions in both diversity and performance.