RuleSmith: Multi-Agent LLMs for Automated Game Balancing

TL;DR

RuleSmith leverages multi-agent LLMs, game engines, and Bayesian optimization to automate game balancing, demonstrating effective convergence to balanced configurations in a complex civilization-style game.

Contribution

This work introduces RuleSmith, the first framework combining multi-agent LLMs, game simulation, and Bayesian optimization for automated game balancing.

Findings

RuleSmith achieves highly balanced game configurations.

The framework provides interpretable rule adjustments.

LLM simulation effectively replaces manual balancing processes.

Abstract

Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, the first framework that achieves automated game balancing by leveraging the reasoning capabilities of multi-agent LLMs. It couples a game engine, multi-agent LLMs self-play, and Bayesian optimization operating over a multi-dimensional rule space. As a proof of concept, we instantiate RuleSmith on CivMini, a simplified civilization-style game containing heterogeneous factions, economy systems, production rules, and combat mechanics, all governed by tunable parameters. LLM agents interpret textual rulebooks and game states to generate actions, to conduct fast evaluation of balance metrics such as win-rate disparities. To search the parameter landscape efficiently, we integrate Bayesian optimization with acquisition-based adaptive sampling and discrete projection: promising candidates receive more evaluation games for accurate assessment, while exploratory candidates receive fewer games for efficient exploration. Experiments show that RuleSmith converges to highly balanced configurations and provides interpretable rule adjustments that can be directly applied to downstream game systems. Our results illustrate that LLM simulation can serve as a powerful surrogate for automating design and balancing in complex multi-agent environments.