Hierarchical Decision Making by Generating and Following Natural Language Instructions

TL;DR

This paper presents a hierarchical decision-making approach that uses natural language instructions as a latent, compositional representation for complex actions, demonstrating improved performance in a real-time strategy game environment.

Contribution

It introduces a novel method of using latent natural language instructions for hierarchical decision making and provides a new dataset from human gameplay for training and evaluation.

Findings

Models using natural language instructions outperform direct imitation models.

The compositional structure of language enhances decision-making effectiveness.

The approach is validated in a complex real-time strategy game environment.

Abstract

We explore using latent natural language instructions as an expressive and compositional representation of complex actions for hierarchical decision making. Rather than directly selecting micro-actions, our agent first generates a latent plan in natural language, which is then executed by a separate model. We introduce a challenging real-time strategy game environment in which the actions of a large number of units must be coordinated across long time scales. We gather a dataset of 76 thousand pairs of instructions and executions from human play, and train instructor and executor models. Experiments show that models using natural language as a latent variable significantly outperform models that directly imitate human actions. The compositional structure of language proves crucial to its effectiveness for action representation. We also release our code, models and data.