Characterising Interventions in Causal Games

TL;DR

This paper extends causal games by defining a comprehensive set of primitive interventions, enabling analysis of complex interventional queries in multi-agent systems, with applications to safe AI design.

Contribution

It introduces a complete set of primitive causal interventions for multi-agent causal games, relaxing previous chronological constraints and facilitating complex causal analysis.

Findings

Defined a sound and complete set of primitive interventions.

Enabled analysis of arbitrarily complex interventional queries.

Applied framework to causal mechanism design and AI safety.

Abstract

Causal games are probabilistic graphical models that enable causal queries to be answered in multi-agent settings. They extend causal Bayesian networks by specifying decision and utility variables to represent the agents' degrees of freedom and objectives. In multi-agent settings, whether each agent decides on their policy before or after knowing the causal intervention is important as this affects whether they can respond to the intervention by adapting their policy. Consequently, previous work in causal games imposed chronological constraints on permissible interventions. We relax this by outlining a sound and complete set of primitive causal interventions so the effect of any arbitrarily complex interventional query can be studied in multi-agent settings. We also demonstrate applications to the design of safe AI systems by considering causal mechanism design and commitment.