LLM-augmented empirical game theoretic simulation for social-ecological systems

TL;DR

This paper compares four LLM-augmented simulation frameworks for social-ecological systems, revealing significant behavioral differences and emphasizing the importance of methodological diversity and expert guidance.

Contribution

It introduces and evaluates four integrated LLM-augmented modeling approaches for social-ecological systems, highlighting their differences and effectiveness.

Findings

Procedural ABMs, generative ABMs, and LLM-EGTA produce different behavioral patterns.

Expert-guided LLM-EGTA with parameterized payoffs is more effective than system prompts.

Methodological diversity enhances understanding of social-ecological dynamics.

Abstract

Designing institutions for social-ecological systems requires models that capture heterogeneity, uncertainty, and strategic interaction. Multiple modeling approaches have emerged to meet this challenge, including empirical game-theoretic analysis (EGTA), which merges ABM's scale and diversity with game-theoretic models' formal equilibrium analysis. The newly popular class of LLM-driven simulations provides yet another approach, and it is not clear how these approaches can be integrated with one another, nor whether the resulting simulations produce a plausible range of behaviours for real-world social-ecological governance. To address this gap, we compare four LLM-augmented frameworks: procedural ABMs, generative ABMs, LLM-EGTA, and expert guided LLM-EGTA, and evaluate them on a real-world case study of irrigation and fishing in the Amu Darya basin under centralized and decentralized governance. Our results show: first, procedural ABMs, generative ABMs, and LLM-augmented EGTA models produce strikingly different patterns of collective behaviour, highlighting the value of methodological diversity. Second, inducing behaviour through system prompts in LLMs is less effective than shaping behaviour through parameterized payoffs in an expert-guided EGTA-based model.