Distributed Multi-Agent Deep Reinforcement Learning for Robust Coordination against Noise

TL;DR

This paper introduces DA3-X, a distributed attentional actor architecture for multi-agent systems, enabling agents to learn cooperatively in noisy environments and outperform baseline methods.

Contribution

The paper proposes a novel distributed attention-based model for multi-agent reinforcement learning that enhances robustness against environmental noise.

Findings

Agents with DA3-X outperform baseline agents in noisy settings

Heatmap visualizations reveal how attention weights adapt to noise

DA3-X enables selective learning and cooperative behavior in noisy environments

Abstract

In multi-agent systems, noise reduction techniques are important for improving the overall system reliability as agents are required to rely on limited environmental information to develop cooperative and coordinated behaviors with the surrounding agents. However, previous studies have often applied centralized noise reduction methods to build robust and versatile coordination in noisy multi-agent environments, while distributed and decentralized autonomous agents are more plausible for real-world application. In this paper, we introduce a \emph{distributed attentional actor architecture model for a multi-agent system} (DA3-X), using which we demonstrate that agents with DA3-X can selectively learn the noisy environment and behave cooperatively. We experimentally evaluate the effectiveness of DA3-X by comparing learning methods with and without DA3-X and show that agents with DA3-X can achieve better performance than baseline agents. Furthermore, we visualize heatmaps of \emph{attentional weights} from the DA3-X to analyze how the decision-making process and coordinated behavior are influenced by noise.