Explaining Decentralized Multi-Agent Reinforcement Learning Policies

TL;DR

This paper introduces methods for explaining decentralized multi-agent reinforcement learning policies, focusing on summarizations and query-based explanations to improve user understanding and interaction.

Contribution

It presents novel explanation techniques tailored for decentralized MARL, addressing the gap in interpretability methods for such systems.

Findings

Summarizations effectively capture task and cooperation dynamics.

Explanations improve user question-answering performance.

Approach is generalizable across domains and algorithms.

Abstract

Multi-Agent Reinforcement Learning (MARL) has gained significant interest in recent years, enabling sequential decision-making across multiple agents in various domains. However, most existing explanation methods focus on centralized MARL, failing to address the uncertainty and nondeterminism inherent in decentralized settings. We propose methods to generate policy summarizations that capture task ordering and agent cooperation in decentralized MARL policies, along with query-based explanations for When, Why Not, and What types of user queries about specific agent behaviors. We evaluate our approach across four MARL domains and two decentralized MARL algorithms, demonstrating its generalizability and computational efficiency. User studies show that our summarizations and explanations significantly improve user question-answering performance and enhance subjective ratings on metrics such as understanding and satisfaction.