From Gameplay Traces to Game Mechanics: Causal Induction with Large Language Models

TL;DR

This paper explores how Large Language Models can reverse-engineer game mechanics from gameplay data by inferring causal models, leading to more accurate and consistent game descriptions than direct code generation methods.

Contribution

It introduces a two-stage causal induction approach using LLMs to infer game rules from gameplay traces, improving accuracy over direct generation methods.

Findings

SCM-based approach outperforms direct code generation in accuracy

Preference win rates reach up to 81% in blind evaluations

Learned SCMs enable causal reinforcement learning and game generation

Abstract

Deep learning agents can achieve high performance in complex game domains without often understanding the underlying causal game mechanics. To address this, we investigate Causal Induction: the ability to infer governing laws from observational data, by tasking Large Language Models (LLMs) with reverse-engineering Video Game Description Language (VGDL) rules from gameplay traces. To reduce redundancy, we select nine representative games from the General Video Game AI (GVGAI) framework using semantic embeddings and clustering. We compare two approaches to VGDL generation: direct code generation from observations, and a two-stage method that first infers a structural causal model (SCM) and then translates it into VGDL. Both approaches are evaluated across multiple prompting strategies and controlled context regimes, varying the amount and form of information provided to the model, from just raw gameplay observations to partial VGDL specifications. Results show that the SCM-based approach more often produces VGDL descriptions closer to the ground truth than direct generation, achieving preference win rates of up to 81\% in blind evaluations and yielding fewer logically inconsistent rules. These learned SCMs can be used for downstream use cases such as causal reinforcement learning, interpretable agents, and procedurally generating novel but logically consistent games.