Contracting with a Learning Agent

TL;DR

This paper studies repeated contracts with learning agents achieving no-regret outcomes, proposing a simple dynamic contract structure that optimizes principal's reward and benefits both players over static contracts.

Contribution

It introduces an optimal dynamic contract solution for repeated principal-agent interactions with no-regret learning agents, extending to non-linear contracts and addressing horizon knowledge.

Findings

Optimal dynamic contract involves switching from a positive scalar to zero, enabling the principal to gain rewards.

The proposed contract structure benefits both principal and agent compared to static contracts.

Results extend to non-linear contracts and consider the impact of horizon knowledge.

Abstract

Many real-life contractual relations differ completely from the clean, static model at the heart of principal-agent theory. Typically, they involve repeated strategic interactions of the principal and agent, taking place under uncertainty and over time. While appealing in theory, players seldom use complex dynamic strategies in practice, often preferring to circumvent complexity and approach uncertainty through learning. We initiate the study of repeated contracts with a learning agent, focusing on agents who achieve no-regret outcomes. Optimizing against a no-regret agent is a known open problem in general games; we achieve an optimal solution to this problem for a canonical contract setting, in which the agent's choice among multiple actions leads to success/failure. The solution has a surprisingly simple structure: for some $\alpha > 0$, initially offer the agent a linear contract with scalar $\alpha$, then switch to offering a linear contract with scalar $0$. This switch causes the agent to ``free-fall'' through their action space and during this time provides the principal with non-zero reward at zero cost. Despite apparent exploitation of the agent, this dynamic contract can leave \emph{both} players better off compared to the best static contract. Our results generalize beyond success/failure, to arbitrary non-linear contracts which the principal rescales dynamically. Finally, we quantify the dependence of our results on knowledge of the time horizon, and are the first to address this consideration in the study of strategizing against learning agents.