Dynamic Sight Range Selection in Multi-Agent Reinforcement Learning

TL;DR

This paper introduces a Dynamic Sight Range Selection (DSR) method for multi-agent reinforcement learning that adaptively adjusts agents' sight ranges during training, improving performance, training speed, and interpretability across various environments and algorithms.

Contribution

The paper presents a novel DSR method that uses UCB to dynamically optimize agents' sight ranges, addressing the sight range dilemma in MARL without relying on global information.

Findings

DSR improves performance in LBF, RWARE, and SMAC environments.

DSR enhances multiple MARL algorithms like QMIX and MAPPO.

DSR accelerates training and offers interpretability of sight ranges.

Abstract

Multi-agent reinforcement Learning (MARL) is often challenged by the sight range dilemma, where agents either receive insufficient or excessive information from their environment. In this paper, we propose a novel method, called Dynamic Sight Range Selection (DSR), to address this issue. DSR utilizes an Upper Confidence Bound (UCB) algorithm and dynamically adjusts the sight range during training. Experiment results show several advantages of using DSR. First, we demonstrate using DSR achieves better performance in three common MARL environments, including Level-Based Foraging (LBF), Multi-Robot Warehouse (RWARE), and StarCraft Multi-Agent Challenge (SMAC). Second, our results show that DSR consistently improves performance across multiple MARL algorithms, including QMIX and MAPPO. Third, DSR offers suitable sight ranges for different training steps, thereby accelerating the training process. Finally, DSR provides additional interpretability by indicating the optimal sight range used during training. Unlike existing methods that rely on global information or communication mechanisms, our approach operates solely based on the individual sight ranges of agents. This approach offers a practical and efficient solution to the sight range dilemma, making it broadly applicable to real-world complex environments.