Toward Policy Explanations for Multi-Agent Reinforcement Learning

TL;DR

This paper introduces new methods for explaining multi-agent reinforcement learning policies, focusing on cooperation summaries and language explanations, which enhance transparency and user satisfaction in complex multi-agent systems.

Contribution

The paper presents novel scalable explanation techniques for MARL, including policy summaries and language explanations, addressing a gap in multi-agent transparency methods.

Findings

Explanations improve user performance in MARL tasks.

Generated explanations increase user satisfaction.

Methods are scalable across different MARL domains.

Abstract

Advances in multi-agent reinforcement learning (MARL) enable sequential decision making for a range of exciting multi-agent applications such as cooperative AI and autonomous driving. Explaining agent decisions is crucial for improving system transparency, increasing user satisfaction, and facilitating human-agent collaboration. However, existing works on explainable reinforcement learning mostly focus on the single-agent setting and are not suitable for addressing challenges posed by multi-agent environments. We present novel methods to generate two types of policy explanations for MARL: (i) policy summarization about the agent cooperation and task sequence, and (ii) language explanations to answer queries about agent behavior. Experimental results on three MARL domains demonstrate the scalability of our methods. A user study shows that the generated explanations significantly improve user performance and increase subjective ratings on metrics such as user satisfaction.