SC-Phi2: A Fine-tuned Small Language Model for StarCraft II Macromanagement Tasks

TL;DR

This paper presents SC-Phi2, a small, fine-tuned language model for StarCraft II macromanagement tasks, combining text and visual data, trained efficiently on limited hardware, and demonstrating strong performance in game strategy tasks.

Contribution

The paper introduces SC-Phi2, a novel small language model specifically fine-tuned for StarCraft II, integrating visual and textual data for strategic game management.

Findings

Effective in build order prediction

Performs well in global state assessment

Trained efficiently on a single GPU

Abstract

This paper introduces SC-Phi2, a fine-tuned StarCraft II small language model for macromanagement tasks. Small language models, like Phi2, Gemma, and DistilBERT, are streamlined versions of large language models (LLMs) with fewer parameters that require less power and memory to run. To teach Microsoft's Phi2 model about StarCraft, we create a new SC2 text dataset with information about StarCraft races, roles, and actions and use it to fine-tune Phi-2 with self-supervised learning. We pair this language model with a Vision Transformer (ViT) from the pre-trained BLIP-2 (Bootstrapping Language Image Pre-training) model, fine-tuning it on the MSC replay dataset. This enables us to construct dynamic prompts that include visual game state information. Unlike the large models used in StarCraft LLMs such as GPT-3.5, Phi2 is trained primarily on textbook data and contains little inherent knowledge of StarCraft II beyond what is provided by our training process. By using LoRA (Low-rank Adaptation) and quantization, our model can be trained on a single GPU. We demonstrate that our model performs well at micromanagement tasks such as build order and global state prediction with a small number of parameters.