Learning from Synthetic Labs: Language Models as Auction Participants

TL;DR

This paper demonstrates that large language models can simulate auction behaviors consistent with economic theory and human patterns, providing a cost-effective framework for auction research and design using LLMs as proxies.

Contribution

It introduces a novel synthetic data-generating process using LLMs for auction studies, showing their ability to replicate classic auction behaviors and serve as flexible, low-cost experimental proxies.

Findings

LLMs with chain of thought reasoning align with experimental auction results

LLMs exhibit risk-averse behavior similar to humans in auctions

Proper prompting improves LLM predictions towards theoretical models

Abstract

This paper investigates the behavior of simulated AI agents (large language models, or LLMs) in auctions, introducing a novel synthetic data-generating process to help facilitate the study and design of auctions. We find that LLMs -- when endowed with chain of thought reasoning capacity -- agree with the experimental literature in auctions across a variety of classic auction formats. In particular, we find that LLM bidders produce results consistent with risk-averse human bidders; that they perform closer to theoretical predictions in obviously strategy-proof auctions; and, that they succumb to the winner's curse in common value settings. On prompting, we find that LLMs are not very sensitive to naive changes in prompts (e.g., language, currency) but can improve dramatically towards theoretical predictions with the right mental model (i.e., the language of Nash deviations). We run 1,000$+$ auctions for less than $\$$400 with GPT-4 models (three orders of magnitude cheaper than modern auction experiments) and develop a framework flexible enough to run auction experiments with any LLM model and a wide range of auction design specifications, facilitating further experimental study by decreasing costs and serving as a proof-of-concept for the use of LLM proxies.