Boardwalk: Towards a Framework for Creating Board Games with LLMs

TL;DR

This paper explores using large language models to automatically generate playable digital versions of board games from natural language rules, aiming to streamline game development.

Contribution

It introduces Boardwalk, a framework for LLM-based board game code generation, and evaluates multiple models' ability to implement diverse games accurately.

Findings

Claude 3.7 Sonnet achieved 55.6% error-free implementations.

API compliance increased error frequency, but error severity depended on the LLM.

The approach demonstrates viability for automated board game coding.

Abstract

Implementing board games in code can be a time-consuming task. However, Large Language Models (LLMs) have been proven effective at generating code for domain-specific tasks with simple contextual information. We aim to investigate whether LLMs can implement digital versions of board games from rules described in natural language. This would be a step towards an LLM-assisted framework for quick board game code generation. We expect to determine the main challenges for LLMs to implement the board games, and how different approaches and models compare to one another. We task three state-of-the-art LLMs (Claude, DeepSeek and ChatGPT) with coding a selection of 12 popular and obscure games in free-form and within Boardwalk, our proposed General Game Playing API. We anonymize the games and components to avoid evoking pre-trained LLM knowledge. The implementations are tested for playability and rule compliance. We evaluate success rate and common errors across LLMs and game popularity. Our approach proves viable, with the best performing model, Claude 3.7 Sonnet, yielding 55.6\% of games without any errors. While compliance with the API increases error frequency, the severity of errors is more significantly dependent on the LLM. We outline future steps for creating a framework to integrate this process, making the elaboration of board games more accessible.