Learning to Incentivize in Repeated Principal-Agent Problems with Adversarial Agent Arrivals

TL;DR

This paper studies a repeated principal-agent problem with adversarial agent arrivals, proposing algorithms with sublinear regret bounds under different assumptions about agent responses, and extending to multiple incentives per round.

Contribution

It introduces the first algorithms with provable regret bounds for adversarial, repeated principal-agent problems with unknown agent behaviors and multiple incentives.

Findings

Achieves $O(rac{ ext{polylog}(N)}{ ext{poly}(T)})$ regret bounds under known greedy responses.

Establishes lower bounds matching the upper bounds up to logarithmic factors.

Extends algorithms to incentivize multiple arms simultaneously in each round.

Abstract

We initiate the study of a repeated principal-agent problem over a finite horizon $T$, where a principal sequentially interacts with $K\geq 2$ types of agents arriving in an adversarial order. At each round, the principal strategically chooses one of the $N$ arms to incentivize for an arriving agent of unknown type. The agent then chooses an arm based on its own utility and the provided incentive, and the principal receives a corresponding reward. The objective is to minimize regret against the best incentive in hindsight. Without prior knowledge of agent behavior, we show that the problem becomes intractable, leading to linear regret. We analyze two key settings where sublinear regret is achievable. In the first setting, the principal knows the arm each agent type would select greedily for any given incentive. Under this setting, we propose an algorithm that achieves a regret bound of $O(\min\{\sqrt{KT\log N},K\sqrt{T}\})$ and provide a matching lower bound up to a $\log K$ factor. In the second setting, an agent's response varies smoothly with the incentive and is governed by a Lipschitz constant $L\geq 1$. Under this setting, we show that there is an algorithm with a regret bound of $\tilde{O}((LN)^{1/3}T^{2/3})$ and establish a matching lower bound up to logarithmic factors. Finally, we extend our algorithmic results for both settings by allowing the principal to incentivize multiple arms simultaneously in each round.