Approximating Auction Equilibria with Reinforcement Learning

TL;DR

This paper presents a reinforcement learning framework using self-play and advanced algorithms to approximate equilibria in complex, multi-item, and dynamic auctions, overcoming computational challenges of traditional methods.

Contribution

It introduces a novel RL-based approach employing Proximal Policy Optimization and Neural Fictitious Self-Play for auction equilibrium approximation.

Findings

Learns robust bidding strategies in various auction settings.

Handles high-dimensional information and delayed payoffs.

Achieves near-optimal strategies comparable to known equilibria.

Abstract

Traditional methods for computing equilibria in auctions become computationally intractable as auction complexity increases, particularly in multi-item and dynamic auctions. This paper introduces a self-play based reinforcement learning approach that employs advanced algorithms such as Proximal Policy Optimization and Neural Fictitious Self-Play to approximate Bayes-Nash equilibria. This framework allows for continuous action spaces, high-dimensional information states, and delayed payoffs. Through self-play, these algorithms can learn robust and near-optimal bidding strategies in auctions with known equilibria, including those with symmetric and asymmetric valuations, private and interdependent values, and multi-round auctions.