Algorithmic Collusion at Test Time: A Meta-game Design and Evaluation

TL;DR

This paper introduces a meta-game framework to analyze the emergence of algorithmic collusion at test time, evaluating how different strategies and settings influence collusion feasibility and strategic behavior.

Contribution

It proposes a novel meta-game design for studying algorithmic collusion, incorporating pretrained policies and in-game adaptation, and provides empirical analysis of strategic interactions.

Findings

Collusion can emerge under certain strategic conditions.

Test-time strategies like reinforcement learning and LLMs influence collusion likelihood.

Strategic relationships are uncovered through empirical best-response graphs.

Abstract

The threat of algorithmic collusion, and whether it merits regulatory intervention, remains debated, as existing evaluations of its emergence often rely on long learning horizons, assumptions about counterparty rationality in adopting collusive strategies, and symmetry in hyperparameters and economic settings among players. To study collusion risk, we introduce a meta-game design for analyzing algorithmic behavior under test-time constraints. We model agents as possessing pretrained policies with distinct strategic characteristics (e.g., competitive, naively cooperative, or robustly collusive), and formulate the problem as selecting a meta-strategy that combines a pretrained, initial policy with an in-game adaptation rule. We seek to examine whether collusion can emerge under rational choices and how agents co-adapt toward cooperation or competition. To this end, we sample normal-form empirical games over meta-strategy profiles, compute relevant game statistics (e.g., payoffs against individuals and regret against an equilibrium mixture of opponents), and construct empirical best-response graphs to uncover strategic relationships. We evaluate reinforcement-learning, UCB, and LLM-based strategies in repeated pricing games under symmetric and asymmetric cost settings, and present findings on the feasibility of algorithmic collusion and the effectiveness of pricing strategies in practical ``test-time'' environments. The source code is available at: https://github.com/chailab-rutgers/CollusionMetagame.