Grammar-based Game Description Generation using Large Language Models

TL;DR

This paper introduces a novel framework using Large Language Models to generate grammatically correct Game Description Language (GDL) descriptions from natural language, improving over baseline methods through iterative refinement.

Contribution

It presents a two-stage approach that first generates a minimal grammar and then iteratively refines game descriptions guided by grammar, advancing natural language to GDL translation.

Findings

Iterative refinement significantly improves grammatical accuracy

The approach outperforms baseline LLM-based methods

Code availability facilitates reproducibility

Abstract

Game Description Language (GDL) provides a standardized way to express diverse games in a machine-readable format, enabling automated game simulation, and evaluation. While previous research has explored game description generation using search-based methods, generating GDL descriptions from natural language remains a challenging task. This paper presents a novel framework that leverages Large Language Models (LLMs) to generate grammatically accurate game descriptions from natural language. Our approach consists of two stages: first, we gradually generate a minimal grammar based on GDL specifications; second, we iteratively improve the game description through grammar-guided generation. Our framework employs a specialized parser that identifies valid subsequences and candidate symbols from LLM responses, enabling gradual refinement of the output to ensure grammatical correctness. Experimental results demonstrate that our iterative improvement approach significantly outperforms baseline methods that directly use LLM outputs. Our code is available at https://github.com/tsunehiko/ggdg