Counterfactual Reasoning for Causal Responsibility Attribution in Probabilistic Multi-Agent Systems

TL;DR

This paper introduces a formal framework for responsibility attribution in probabilistic multi-agent systems using counterfactual reasoning, Shapley values, and Nash equilibria.

Contribution

It models responsibility in stochastic multi-agent systems, proposes a fairness-preserving responsibility measure, and integrates strategic reasoning with equilibrium analysis.

Findings

Responsibility can be quantified using counterfactual reasoning and Shapley values.

The framework ensures fairness and consistency in responsibility allocation.

Stable strategies balancing responsibility and reward can be computed using Nash equilibrium.

Abstract

Responsibility allocation -- determining the extent to which agents are accountable for outcomes -- is a fundamental challenge in the design and analysis of multi-agent systems. In this work, we model such systems as concurrent stochastic multi-player games and introduce a notion of retrospective (backward) counterfactual responsibility, which quantifies an agent's accountability for outcomes resulting from a given strategy profile. To allocate responsibility among agents, we utilise the Shapley value and formally show that this method satisfies key desirable properties, including fairness and consistency. Building on this foundation, we propose a formal framework that supports both verification and strategic reasoning in responsibility-aware multi-agent systems. Furthermore, by adopting Nash equilibrium as the solution concept, we demonstrate how to compute stable strategy profiles in which agents trade off responsibility against expected reward.