Society of Mind Meets Real-Time Strategy: A Hierarchical Multi-Agent Framework for Strategic Reasoning

TL;DR

This paper introduces HIMA, a hierarchical multi-agent framework using imitation learning and a strategic planner to improve real-time strategic reasoning in complex environments like StarCraft II.

Contribution

The paper presents a novel hierarchical multi-agent framework with specialized imitation agents and a meta-controller, enhancing strategic reasoning and adaptability in complex, partially observable environments.

Findings

HIMA outperforms existing methods in strategic clarity and adaptability.

The framework improves computational efficiency in complex strategic tasks.

HIMA demonstrates robustness across all StarCraft II race matchups.

Abstract

Large Language Models (LLMs) have recently demonstrated impressive action sequence prediction capabilities but often struggle with dynamic, long-horizon tasks such as real-time strategic games. In a game such as StarCraftII (SC2), agents need to manage resource constraints and adapt to evolving battlefield situations in a partially observable environment. This often overwhelms exisiting LLM-based approaches. To address these challenges, we propose a hierarchical multi-agent framework that employs specialized imitation learning agents under a meta-controller called Strategic Planner (SP). By expert demonstrations, each specialized agent learns a distinctive strategy, such as aerial support or defensive maneuvers, and produces coherent, structured multistep action sequences. The SP then orchestrates these proposals into a single, environmentally adaptive plan that ensures local decisions aligning with long-term strategies. We call this HIMA (Hierarchical Imitation Multi-Agent). We also present TEXTSCII-ALL, a comprehensive SC2 testbed that encompasses all race match combinations in SC2. Our empirical results show that HIMA outperforms state of the arts in strategic clarity, adaptability, and computational efficiency, underscoring the potential of combining specialized imitation modules with meta-level orchestration to develop more robust, general-purpose AI agents.