Interpreting Emergent Extreme Events in Multi-Agent Systems

TL;DR

This paper introduces a novel framework using adapted Shapley values to interpret and attribute the origins of extreme events in multi-agent systems, enhancing understanding of their emergence and contributing to system safety.

Contribution

It presents the first method for explaining emergent extreme events in multi-agent systems by attributing causality to agent actions over time using a new adaptation of the Shapley value.

Findings

Effective attribution of extreme events to agent actions across scenarios

Quantitative metrics for characterizing features of extreme events

Insights into the emergence of extreme phenomena in diverse systems

Abstract

Large language model-powered multi-agent systems have emerged as powerful tools for simulating complex human-like systems. The interactions within these systems often lead to extreme events whose origins remain obscured by the black box of emergence. Interpreting these events is critical for system safety. This paper proposes the first framework for explaining emergent extreme events in multi-agent systems, aiming to answer three fundamental questions: When does the event originate? Who drives it? And what behaviors contribute to it? Specifically, we adapt the Shapley value to faithfully attribute the occurrence of extreme events to each action taken by agents at different time steps, i.e., assigning an attribution score to the action to measure its influence on the event. We then aggregate the attribution scores along the dimensions of time, agent, and behavior to quantify the risk contribution of each dimension. Finally, we design a set of metrics based on these contribution scores to characterize the features of extreme events. Experiments across diverse multi-agent system scenarios (economic, financial, and social) demonstrate the effectiveness of our framework and provide general insights into the emergence of extreme phenomena.