Comparing Observation and Action Representations for Deep Reinforcement Learning in $\mu$RTS

TL;DR

This study compares global and local observation and action space representations in deep reinforcement learning for RTS games, finding local representations more effective for resource harvesting tasks.

Contribution

It introduces a comparative analysis of observation and action representations in DRL for RTS, highlighting the advantages of local over global representations.

Findings

Local representation outperforms global in resource harvesting tasks

Evaluation conducted in the $$RTS environment

Preliminary results suggest local observations improve learning efficiency

Abstract

This paper presents a preliminary study comparing different observation and action space representations for Deep Reinforcement Learning (DRL) in the context of Real-time Strategy (RTS) games. Specifically, we compare two representations: (1) a global representation where the observation represents the whole game state, and the RL agent needs to choose which unit to issue actions to, and which actions to execute; and (2) a local representation where the observation is represented from the point of view of an individual unit, and the RL agent picks actions for each unit independently. We evaluate these representations in $\mu$RTS showing that the local representation seems to outperform the global representation when training agents with the task of harvesting resources.