Winning at Any Cost -- Infringing the Cartel Prohibition With Reinforcement Learning

TL;DR

This paper investigates how reinforcement learning agents in pricing scenarios can inadvertently develop collusive behaviors, including tacit cooperation, and proposes methods to detect such collusion to prevent anti-competitive practices.

Contribution

It introduces a modified prisoner's dilemma scenario with reinforcement learning agents to analyze collusion formation and suggests detection strategies for collusive behaviors.

Findings

Agents can develop collusive strategies in pricing games.

Action selection processes can be segmented into stages for analysis.

Tacit cooperation can emerge without explicit training.

Abstract

Pricing decisions are increasingly made by AI. Thanks to their ability to train with live market data while making decisions on the fly, deep reinforcement learning algorithms are especially effective in taking such pricing decisions. In e-commerce scenarios, multiple reinforcement learning agents can set prices based on their competitor's prices. Therefore, research states that agents might end up in a state of collusion in the long run. To further analyze this issue, we build a scenario that is based on a modified version of a prisoner's dilemma where three agents play the game of rock paper scissors. Our results indicate that the action selection can be dissected into specific stages, establishing the possibility to develop collusion prevention systems that are able to recognize situations which might lead to a collusion between competitors. We furthermore provide evidence for a situation where agents are capable of performing a tacit cooperation strategy without being explicitly trained to do so.