A Visual Analytics System to Understand Behaviors of Multi Agents in Reinforcement Learning

TL;DR

This paper introduces MARLViz, a visual analytics system designed to help researchers understand complex multi-agent interactions and policies in reinforcement learning environments through visual analysis.

Contribution

The paper presents MARLViz, a novel visual analytics tool that visualizes agent behaviors and interactions in multi-agent reinforcement learning, addressing limitations of existing analysis methods.

Findings

MARLViz effectively visualizes differences in agent behaviors under various settings.

The system helps identify and interpret complex interaction patterns among agents.

Analysis of scenarios reveals insights into agent strategies and cooperation.

Abstract

Multi-Agent Reinforcement Learning (MARL) is a branch of machine learning in which agents interact and learn optimal policies through trial and error, addressing complex scenarios where multiple agents interact and learn in the same environment at the same time. Analyzing and understanding these complex interactions is challenging, and existing analysis methods are limited in their ability to fully reflect and interpret this complexity. To address these challenges, we provide MARLViz, a visual analytics system for visualizing and analyzing the policies and interactions of agents in MARL environments. The system is designed to visually show the difference in behavior of agents under different environment settings and help users understand complex interaction patterns. In this study, we analyzed agents with similar behaviors and selected scenarios to understand the interactions of the agents, which made it easier to understand the strategies of agents in MARL.