DIML: Differentiable Inverse Mechanism Learning from Behaviors of Multi-Agent Learning Trajectories

TL;DR

DIML introduces a likelihood-based framework for recovering unknown incentive mechanisms from observed multi-agent learning behaviors, enabling counterfactual analysis and scalable inference in complex strategic environments.

Contribution

We propose DIML, a novel differentiable inverse mechanism learning method that infers unstructured incentive mechanisms from observed behaviors, with proven identifiability and scalability.

Findings

DIML accurately recovers incentive differences in simulated environments.

It supports reliable counterfactual predictions in large-scale multi-agent settings.

Performance rivals oracle methods in small environments and scales to hundreds of agents.

Abstract

We study inverse mechanism learning: recovering an unknown incentive-generating mechanism from observed strategic interaction traces of self-interested learning agents. Unlike inverse game theory and multi-agent inverse reinforcement learning, which typically infer utility/reward parameters inside a structured mechanism, our target includes unstructured mechanism -- a (possibly neural) mapping from joint actions to per-agent payoffs. Unlike differentiable mechanism design, which optimizes mechanisms forward, we infer mechanisms from behavior in an observational setting. We propose DIML, a likelihood-based framework that differentiates through a model of multi-agent learning dynamics and uses the candidate mechanism to generate counterfactual payoffs needed to predict observed actions. We establish identifiability of payoff differences under a conditional logit response model and prove statistical consistency of maximum likelihood estimation under standard regularity conditions. We evaluate DIML with simulated interactions of learning agents across unstructured neural mechanisms, congestion tolling, public goods subsidies, and large-scale anonymous games. DIML reliably recovers identifiable incentive differences and supports counterfactual prediction, where its performance rivals tabular enumeration oracle in small environments and its convergence scales to large, hundred-participant environments. Code to reproduce our experiments is open-sourced.