Reinforcement Learning of Sequential Price Mechanisms

TL;DR

This paper applies reinforcement learning to optimize sequential price mechanisms, a broad class of strategyproof mechanisms, demonstrating its effectiveness through experimental results.

Contribution

It introduces a reinforcement learning framework for designing optimal sequential price mechanisms, expanding the applicability of RL in mechanism design.

Findings

RL can learn near-optimal mechanisms in various settings

Sequential price mechanisms can outperform static mechanisms under certain conditions

Deep policies are sometimes necessary for optimality

Abstract

We introduce the use of reinforcement learning for indirect mechanisms, working with the existing class of sequential price mechanisms, which generalizes both serial dictatorship and posted price mechanisms and essentially characterizes all strongly obviously strategyproof mechanisms. Learning an optimal mechanism within this class forms a partially-observable Markov decision process. We provide rigorous conditions for when this class of mechanisms is more powerful than simpler static mechanisms, for sufficiency or insufficiency of observation statistics for learning, and for the necessity of complex (deep) policies. We show that our approach can learn optimal or near-optimal mechanisms in several experimental settings.