Adaptive Incentive Design with Learning Agents

TL;DR

This paper introduces an adaptive incentive mechanism that learns optimal incentives in dynamic environments with learning agents, ensuring convergence to socially optimal strategies across various game types.

Contribution

It presents a novel two-timescale adaptive incentive mechanism that is agnostic to players' learning dynamics and guarantees convergence to social optima.

Findings

Mechanism converges to fixed points representing optimal incentives.

Applicable to both atomic and non-atomic games.

Validated in aggregative and routing game scenarios.

Abstract

We propose an adaptive incentive mechanism that learns the optimal incentives in environments where players continuously update their strategies. Our mechanism updates incentives based on each player's externality, defined as the difference between the player's marginal cost and the operator's marginal cost at each time step. The proposed mechanism updates the incentives on a slower timescale compared to the players' learning dynamics, resulting in a two-timescale coupled dynamical system. Notably, this mechanism is agnostic to the specific learning dynamics used by players to update their strategies. We show that any fixed point of this adaptive incentive mechanism corresponds to the optimal incentive mechanism, ensuring that the Nash equilibrium coincides with the socially optimal strategy. Additionally, we provide sufficient conditions under which the adaptive mechanism converges to a fixed point. Our results apply to both atomic and non-atomic games. To demonstrate the effectiveness of our proposed mechanism, we verify the convergence conditions in two practically relevant classes of games: atomic aggregative games and non-atomic routing games.