SC2Arena and StarEvolve: Benchmark and Self-Improvement Framework for LLMs in Complex Decision-Making Tasks

TL;DR

This paper introduces SC2Arena, a comprehensive StarCraft II benchmark, and StarEvolve, a hierarchical self-improvement framework for LLMs to enhance strategic decision-making in complex environments.

Contribution

It presents a full-scale StarCraft II benchmark supporting all playable races and actions, and a novel self-improvement framework for LLMs with iterative fine-tuning and strategic planning.

Findings

StarEvolve outperforms existing models in strategic planning tasks.

SC2Arena enables more realistic and comprehensive evaluation of LLMs.

The framework facilitates continuous self-improvement of AI agents.

Abstract

Evaluating large language models (LLMs) in complex decision-making is essential for advancing AI's ability for strategic planning and real-time adaptation. However, existing benchmarks for tasks like StarCraft II fail to capture the game's full complexity, such as its complete game context, diverse action spaces, and all playable races. To address this gap, we present SC2Arena, a benchmark that fully supports all playable races, low-level action spaces, and optimizes text-based observations to tackle spatial reasoning challenges. Complementing this, we introduce StarEvolve, a hierarchical framework that integrates strategic planning with tactical execution, featuring iterative self-correction and continuous improvement via fine-tuning on high-quality gameplay data. Its key components include a Planner-Executor-Verifier structure to break down gameplay, and a scoring system for selecting high-quality training samples. Comprehensive analysis using SC2Arena provides valuable insights into developing generalist agents that were not possible with previous benchmarks. Experimental results also demonstrate that our proposed StarEvolve achieves superior performance in strategic planning. Our code, environment, and algorithms are publicly available.