Online Learning for Dynamic Vickrey-Clarke-Groves Mechanism in Unknown Environments

TL;DR

This paper develops an online reinforcement learning approach for implementing a dynamic VCG mechanism in unknown, evolving auction environments, ensuring desirable economic properties with performance guarantees.

Contribution

It extends the VCG mechanism to a dynamic, unknown environment and proposes a reinforcement learning algorithm to learn and implement this mechanism online.

Findings

The proposed algorithm achieves near-optimal performance with regret guarantees.

The dynamic VCG mechanism maintains efficiency, truthfulness, and individual rationality approximately.

Experimental results demonstrate effective learning in unknown auction environments.

Abstract

We consider the problem of online dynamic mechanism design for sequential auctions in unknown environments, where the underlying market and, thus, the bidders' values vary over time as interactions between the seller and the bidders progress. We model the sequential auctions as an infinite-horizon average-reward Markov decision process (MDP). In each round, the seller determines an allocation and sets a payment for each bidder, while each bidder receives a private reward and submits a sealed bid to the seller. The state, which represents the underlying market, evolves according to an unknown transition kernel and the seller's allocation policy without episodic resets. We first extend the Vickrey-Clarke-Groves (VCG) mechanism to sequential auctions, thereby obtaining a dynamic counterpart that preserves the desired properties: efficiency, truthfulness, and individual rationality. We then focus on the online setting and develop a reinforcement learning algorithm for the seller to learn the underlying MDP and implement a mechanism that closely resembles the dynamic VCG mechanism. We show that the learned mechanism approximately satisfies efficiency, truthfulness, and individual rationality and achieves guaranteed performance in terms of various notions of regret.