Robust Communicative Multi-Agent Reinforcement Learning with Active Defense

TL;DR

This paper introduces ADMAC, an active defense framework for multi-agent reinforcement learning that estimates message reliability and adjusts their influence to enhance robustness against adversarial attacks.

Contribution

It proposes an active defense strategy with a novel framework, ADMAC, that improves robustness in communicative MARL by estimating message reliability and adjusting their impact.

Findings

ADMAC outperforms existing methods under various attack types.

The framework effectively balances communication performance and robustness.

Experimental results validate the superiority of ADMAC in three tasks.

Abstract

Communication in multi-agent reinforcement learning (MARL) has been proven to effectively promote cooperation among agents recently. Since communication in real-world scenarios is vulnerable to noises and adversarial attacks, it is crucial to develop robust communicative MARL technique. However, existing research in this domain has predominantly focused on passive defense strategies, where agents receive all messages equally, making it hard to balance performance and robustness. We propose an active defense strategy, where agents automatically reduce the impact of potentially harmful messages on the final decision. There are two challenges to implement this strategy, that are defining unreliable messages and adjusting the unreliable messages' impact on the final decision properly. To address them, we design an Active Defense Multi-Agent Communication framework (ADMAC), which estimates the reliability of received messages and adjusts their impact on the final decision accordingly with the help of a decomposable decision structure. The superiority of ADMAC over existing methods is validated by experiments in three communication-critical tasks under four types of attacks.