Practical Performative Policy Learning with Strategic Agents

TL;DR

This paper introduces a scalable, assumption-light method for performative policy learning in strategic environments, leveraging causal inference and gradient optimization to handle endogenous distribution shifts efficiently.

Contribution

It relaxes parametric assumptions in strategic policy learning, uncovers a low-dimensional structure in distribution shifts, and proposes a gradient-based algorithm with theoretical guarantees.

Findings

High sample efficiency compared to bandit-based methods

Effective handling of high-dimensional data shifts

Successful application in complex strategic environments

Abstract

This paper studies the performative policy learning problem, where agents adjust their features in response to a released policy to improve their potential outcomes, inducing an endogenous distribution shift. There has been growing interest in training machine learning models in strategic environments, including strategic classification and performative prediction. However, existing approaches often rely on restrictive parametric assumptions: micro-level utility models in strategic classification and macro-level data distribution maps in performative prediction, severely limiting scalability and generalizability. We approach this problem as a complex causal inference task, relaxing parametric assumptions on both micro-level agent behavior and macro-level data distribution. Leveraging bounded rationality, we uncover a practical low-dimensional structure in distribution shifts and construct an effective mediator in the causal path from the deployed model to the shifted data. We then propose a gradient-based policy optimization algorithm with a differentiable classifier as a substitute for the high-dimensional distribution map. Our algorithm efficiently utilizes batch feedback and limited manipulation patterns. Our approach achieves high sample efficiency compared to methods reliant on bandit feedback or zero-order optimization. We also provide theoretical guarantees for algorithmic convergence. Extensive and challenging experiments on high-dimensional settings demonstrate our method's practical efficacy.