Learning Collusion in Episodic, Inventory-Constrained Markets

TL;DR

This paper investigates how deep reinforcement learning agents can learn to tacitly collude in complex, inventory-constrained markets with perishable goods, extending prior work to more realistic market scenarios.

Contribution

It formalizes collusion in inventory-constrained markets, introduces a new metric for collusive behavior, and demonstrates that RL agents can learn to collude in these settings.

Findings

RL agents can learn collusive strategies in complex markets

A new metric effectively measures collusion levels

Efficient computational methods derive equilibrium price levels

Abstract

Pricing algorithms have demonstrated the capability to learn tacit collusion that is largely unaddressed by current regulations. Their increasing use in markets, including oligopolistic industries with a history of collusion, calls for closer examination by competition authorities. In this paper, we extend the study of tacit collusion in learning algorithms from basic pricing games to more complex markets characterized by perishable goods with fixed supply and sell-by dates, such as airline tickets, perishables, and hotel rooms. We formalize collusion within this framework and introduce a metric based on price levels under both the competitive (Nash) equilibrium and collusive (monopolistic) optimum. Since no analytical expressions for these price levels exist, we propose an efficient computational approach to derive them. Through experiments, we demonstrate that deep reinforcement learning agents can learn to collude in this more complex domain. Additionally, we analyze the underlying mechanisms and structures of the collusive strategies these agents adopt.