Not Yet: Humans Outperform LLMs in a Colonel Blotto Tournament
Dmitry Dagaev, Egor Ivanov, Petr Parshakov, Alexey Savvateev, Gleb Vasiliev

TL;DR
This study compares human and LLM strategies in the Colonel Blotto game, revealing humans outperform LLMs due to better-calibrated heuristics and strategic reasoning, with minimal strategy adjustment across tournaments.
Contribution
First comprehensive tournament-based comparison of human and LLM strategies in a complex game, highlighting the importance of strategic sophistication and reasoning depth.
Findings
Humans outperform LLMs in strategic allocation.
Strategic sophistication and reasoning depth are crucial for success.
Humans show minimal strategy adjustment across different opponents.
Abstract
The emergence of large language models (LLMs) has spurred economists to study how humans and LLMs behave in strategic settings. We organized a series of round-robin tournaments in the Colonel Blotto game. This game attracts game theorists' attention due to high-dimensional action space and the absence of pure strategy Nash equilibria. In the first tournament, more than 200 human participants competed against one another. In the second tournament, several popular LLMs were invited to submit strategies. In the third tournament, we matched the number of LLM strategies to the number submitted by humans. We find that humans more often employ better-calibrated intermediate-level allocation heuristics and outperform the simpler, more stereotyped strategies submitted by LLMs. Strategic sophistication is key to success if and only if the necessary level of reasoning depth is reached, while lower…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
