End-to-End Learning and Intervention in Games

TL;DR

This paper introduces a unified end-to-end framework for learning about agents in games and designing interventions, using differentiable equilibrium computation methods to optimize social outcomes.

Contribution

It proposes a novel framework that integrates equilibrium computation with learning and intervention, employing explicit and implicit differentiation methods for end-to-end optimization.

Findings

Both approaches accurately differentiate through game equilibria.

The framework effectively models real-world social dilemmas.

Conditions for choosing between explicit and implicit methods are established.

Abstract

In a social system, the self-interest of agents can be detrimental to the collective good, sometimes leading to social dilemmas. To resolve such a conflict, a central designer may intervene by either redesigning the system or incentivizing the agents to change their behaviors. To be effective, the designer must anticipate how the agents react to the intervention, which is dictated by their often unknown payoff functions. Therefore, learning about the agents is a prerequisite for intervention. In this paper, we provide a unified framework for learning and intervention in games. We cast the equilibria of games as individual layers and integrate them into an end-to-end optimization framework. To enable the backward propagation through the equilibria of games, we propose two approaches, respectively based on explicit and implicit differentiation. Specifically, we cast the equilibria as the solutions to variational inequalities (VIs). The explicit approach unrolls the projection method for solving VIs, while the implicit approach exploits the sensitivity of the solutions to VIs. At the core of both approaches is the differentiation through a projection operator. Moreover, we establish the correctness of both approaches and identify the conditions under which one approach is more desirable than the other. The analytical results are validated using several real-world problems.