Learning Diverse Policies in MOBA Games via Macro-Goals

TL;DR

This paper introduces MGG, a macro-goals guided framework that enhances policy diversity in MOBA game AI by abstracting strategies as macro-goals from human demonstrations and training a Meta-Controller to predict and guide these goals.

Contribution

The paper presents a novel macro-goals guided framework (MGG) that improves policy diversity in MOBA game AI by leveraging human demonstrations and a Meta-Controller for macro-goal prediction.

Findings

MGG achieves diverse policies across different matches and lineups.

MGG outperforms state-of-the-art methods on Honor of Kings for 102 heroes.

Abstract

Recently, many researchers have made successful progress in building the AI systems for MOBA-game-playing with deep reinforcement learning, such as on Dota 2 and Honor of Kings. Even though these AI systems have achieved or even exceeded human-level performance, they still suffer from the lack of policy diversity. In this paper, we propose a novel Macro-Goals Guided framework, called MGG, to learn diverse policies in MOBA games. MGG abstracts strategies as macro-goals from human demonstrations and trains a Meta-Controller to predict these macro-goals. To enhance policy diversity, MGG samples macro-goals from the Meta-Controller prediction and guides the training process towards these goals. Experimental results on the typical MOBA game Honor of Kings demonstrate that MGG can execute diverse policies in different matches and lineups, and also outperform the state-of-the-art methods over 102 heroes.