4Hammer: a board-game reinforcement learning environment for the hour long time frame

TL;DR

The paper introduces 4Hammer, a complex Warhammer 40,000-based reinforcement learning environment designed to evaluate AI performance on long-duration, rule-intensive board games, addressing a gap in existing RL benchmarks.

Contribution

It presents a new digital twin simulation environment for Warhammer 40,000, enabling RL research on complex, long-duration, rule-based board games.

Findings

First RL environment for Warhammer 40,000

Supports long-duration, complex rule-based gameplay

Facilitates evaluation of AI on detailed natural language rules

Abstract

Large Language Models (LLMs) have demonstrated strong performance on tasks with short time frames, but struggle with tasks requiring longer durations. While datasets covering extended-duration tasks, such as software engineering tasks or video games, do exist, there are currently few implementations of complex board games specifically designed for reinforcement learning and LLM evaluation. To address this gap, we propose the 4Hammer reinforcement learning environment, a digital twin simulation of a subset of Warhammer 40,000-a complex, zero-sum board game. Warhammer 40,000 features intricate rules, requiring human players to thoroughly read and understand over 50 pages of detailed natural language rules, grasp the interactions between their game pieces and those of their opponents, and independently track and communicate the evolving game state.