Synthesis of Reward Machines for Multi-Agent Equilibrium Design (Full Version)

TL;DR

This paper introduces a method for designing dynamic incentive structures, called reward machines, to optimize outcomes in multi-agent games within constrained authority, providing polynomial-time solutions and synthesis techniques.

Contribution

It proposes a novel framework for equilibrium design using reward machines, with polynomial-time algorithms and complexity results for payoff improvement problems.

Findings

Reward machines effectively represent dynamic incentives.

Payoff improvement problem solvable in polynomial time with NP oracle.

Framework extends mechanism design to equilibrium design with constrained authority.

Abstract

Mechanism design is a well-established game-theoretic paradigm for designing games to achieve desired outcomes. This paper addresses a closely related but distinct concept, equilibrium design. Unlike mechanism design, the designer's authority in equilibrium design is more constrained; she can only modify the incentive structures in a given game to achieve certain outcomes without the ability to create the game from scratch. We study the problem of equilibrium design using dynamic incentive structures, known as reward machines. We use weighted concurrent game structures for the game model, with goals (for the players and the designer) defined as mean-payoff objectives. We show how reward machines can be used to represent dynamic incentives that allocate rewards in a manner that optimises the designer's goal. We also introduce the main decision problem within our framework, the payoff improvement problem. This problem essentially asks whether there exists a dynamic incentive (represented by some reward machine) that can improve the designer's payoff by more than a given threshold value. We present two variants of the problem: strong and weak. We demonstrate that both can be solved in polynomial time using a Turing machine equipped with an NP oracle. Furthermore, we also establish that these variants are either NP-hard or coNP-hard. Finally, we show how to synthesise the corresponding reward machine if it exists.