Learning to Reason in Round-based Games: Multi-task Sequence Generation for Purchasing Decision Making in First-person Shooters

TL;DR

This paper introduces a novel multi-task sequence generation model for interpreting round-based purchasing decisions in CS:GO, leveraging meta-learning and comprehensive state representations to enhance reasoning in complex, episodic scenarios.

Contribution

It proposes a new Sequence Reasoner with specialized encoders and multi-task decoding, applying meta-learning to improve interpretability of strategic decisions in round-based games.

Findings

The model outperforms greedy approaches in ablation studies.

State representations effectively capture strategic nuances.

Meta-learning enhances the model's adaptability and reasoning ability.

Abstract

Sequential reasoning is a complex human ability, with extensive previous research focusing on gaming AI in a single continuous game, round-based decision makings extending to a sequence of games remain less explored. Counter-Strike: Global Offensive (CS:GO), as a round-based game with abundant expert demonstrations, provides an excellent environment for multi-player round-based sequential reasoning. In this work, we propose a Sequence Reasoner with Round Attribute Encoder and Multi-Task Decoder to interpret the strategies behind the round-based purchasing decisions. We adopt few-shot learning to sample multiple rounds in a match, and modified model agnostic meta-learning algorithm Reptile for the meta-learning loop. We formulate each round as a multi-task sequence generation problem. Our state representations combine action encoder, team encoder, player features, round attribute encoder, and economy encoders to help our agent learn to reason under this specific multi-player round-based scenario. A complete ablation study and comparison with the greedy approach certify the effectiveness of our model. Our research will open doors for interpretable AI for understanding episodic and long-term purchasing strategies beyond the gaming community.