On the Complexity of Sequential Incentive Design

TL;DR

This paper studies the complexity of designing incentive sequences for agents modeled as Markov decision processes, revealing computational hardness results and proposing algorithms for optimal incentive synthesis.

Contribution

It formalizes the sequential incentive design problem, proves its PSPACE-hardness, identifies NP-complete variants under restrictions, and offers algorithms for optimal solutions.

Findings

BMP is PSPACE-hard to solve.

Certain restrictions lead to NP-complete variants.

Linear programming can solve BMP under specific reward set conditions.

Abstract

In many scenarios, a principal dynamically interacts with an agent and offers a sequence of incentives to align the agent's behavior with a desired objective. This paper focuses on the problem of synthesizing an incentive sequence that, once offered, induces the desired agent behavior even when the agent's intrinsic motivation is unknown to the principal. We model the agent's behavior as a Markov decision process, express its intrinsic motivation as a reward function, which belongs to a finite set of possible reward functions, and consider the incentives as additional rewards offered to the agent. We first show that the behavior modification problem (BMP), i.e., the problem of synthesizing an incentive sequence that induces a desired agent behavior at minimum total cost to the principal, is PSPACE-hard. Moreover, we show that by imposing certain restrictions on the incentive sequences available to the principal, one can obtain two NP-complete variants of the BMP. We also provide a sufficient condition on the set of possible reward functions under which the BMP can be solved via linear programming. Finally, we propose two algorithms to compute globally and locally optimal solutions to the NP-complete variants of the BMP.