On Mechanism Underlying Algorithmic Collusion

TL;DR

This paper investigates the mechanisms behind algorithmic collusion, showing that insufficient learning rather than strategic manipulation drives collusive outcomes, and explores how algorithms adapt and stabilize in simulated markets.

Contribution

It demonstrates that only Nash Equilibrium is stochastically stable in symmetric games, and analyzes how learning dynamics lead to collusive price cycles in simulations.

Findings

Nash Equilibrium is the only stochastically stable outcome in symmetric games.

Algorithms exhibit Edgeworth cycle-like price fluctuations due to learning dynamics.

Lower exploration rates and Q-values can lead to stable, high-price collusion.

Abstract

Two issues of algorithmic collusion are addressed in this paper. First, we show that in a general class of symmetric games, including Prisoner's Dilemma, Bertrand competition, and any (nonlinear) mixture of first and second price auction, only (strict) Nash Equilibrium (NE) is stochastically stable. Therefore, the tacit collusion is driven by failure to learn NE due to insufficient learning, instead of learning some strategies to sustain collusive outcomes. Second, we study how algorithms adapt to collusion in real simulations with insufficient learning. Extensive explorations in early stages and discount factors inflates the Q-value, which interrupts the sequential and alternative price undercut and leads to bilateral rebound. The process is iterated, making the price curves like Edgeworth cycles. When both exploration rate and Q-value decrease, algorithms may bilaterally rebound to relatively high common price level by coincidence, and then get stuck. Finally, we accommodate our reasoning to simulation outcomes in the literature, including optimistic initialization, market design and algorithm design.