Policy Learning with Competing Agents

TL;DR

This paper develops a method for learning treatment policies under capacity constraints when agents respond strategically, using mean-field equilibrium concepts and a consistent policy gradient estimator, demonstrated on educational data.

Contribution

It introduces a novel approach to policy learning with strategic agents under capacity limits, leveraging mean-field equilibrium and a new estimator.

Findings

The threshold for treatment converges to the mean-field equilibrium as the number of agents grows.

The proposed estimator effectively learns policies in strategic, capacity-constrained settings.

Empirical results show successful policy learning on real educational data.

Abstract

Decision makers often aim to learn a treatment assignment policy under a capacity constraint on the number of agents that they can treat. When agents can respond strategically to such policies, competition arises, complicating estimation of the optimal policy. In this paper, we study capacity-constrained treatment assignment in the presence of such interference. We consider a dynamic model where the decision maker allocates treatments at each time step and heterogeneous agents myopically best respond to the previous treatment assignment policy. When the number of agents is large but finite, we show that the threshold for receiving treatment under a given policy converges to the policy's mean-field equilibrium threshold. Based on this result, we develop a consistent estimator for the policy gradient. In a semi-synthetic experiment with data from the National Education Longitudinal Study of 1988, we demonstrate that this estimator can be used for learning capacity-constrained policies in the presence of strategic behavior.