Complex Instruction Following with Diverse Style Policies in Football Games

TL;DR

This paper introduces LCDSP, a new reinforcement learning framework that enables agents in football games to understand high-level instructions and perform diverse behaviors by translating language commands into style parameters.

Contribution

The paper proposes LCDSP, combining style training and interpretation to improve language understanding and behavior diversity in complex multi-agent environments.

Findings

Effective comprehension of abstract instructions in football games.

Accurate translation of language commands into diverse behaviors.

Enhanced performance in complex multi-agent scenarios.

Abstract

Despite advancements in language-controlled reinforcement learning (LC-RL) for basic domains and straightforward commands (e.g., object manipulation and navigation), effectively extending LC-RL to comprehend and execute high-level or abstract instructions in complex, multi-agent environments, such as football games, remains a significant challenge. To address this gap, we introduce Language-Controlled Diverse Style Policies (LCDSP), a novel LC-RL paradigm specifically designed for complex scenarios. LCDSP comprises two key components: a Diverse Style Training (DST) method and a Style Interpreter (SI). The DST method efficiently trains a single policy capable of exhibiting a wide range of diverse behaviors by modulating agent actions through style parameters (SP). The SI is designed to accurately and rapidly translate high-level language instructions into these corresponding SP. Through extensive experiments in a complex 5v5 football environment, we demonstrate that LCDSP effectively comprehends abstract tactical instructions and accurately executes the desired diverse behavioral styles, showcasing its potential for complex, real-world applications.