Towards Comprehensive Testing on the Robustness of Cooperative Multi-agent Reinforcement Learning

TL;DR

This paper introduces MARLSafe, a comprehensive framework for testing the robustness of cooperative multi-agent reinforcement learning algorithms across state, action, and reward aspects, revealing many algorithms are vulnerable.

Contribution

The paper proposes MARLSafe, the first framework to evaluate c-MARL robustness comprehensively from multiple aspects, and develops new attack methods for testing.

Findings

Many state-of-the-art c-MARL algorithms show low robustness.

Existing attacks are limited to single aspects, while MARLSafe considers multiple.

The framework highlights the urgent need to improve c-MARL robustness.

Abstract

While deep neural networks (DNNs) have strengthened the performance of cooperative multi-agent reinforcement learning (c-MARL), the agent policy can be easily perturbed by adversarial examples. Considering the safety critical applications of c-MARL, such as traffic management, power management and unmanned aerial vehicle control, it is crucial to test the robustness of c-MARL algorithm before it was deployed in reality. Existing adversarial attacks for MARL could be used for testing, but is limited to one robustness aspects (e.g., reward, state, action), while c-MARL model could be attacked from any aspect. To overcome the challenge, we propose MARLSafe, the first robustness testing framework for c-MARL algorithms. First, motivated by Markov Decision Process (MDP), MARLSafe consider the robustness of c-MARL algorithms comprehensively from three aspects, namely state robustness, action robustness and reward robustness. Any c-MARL algorithm must simultaneously satisfy these robustness aspects to be considered secure. Second, due to the scarceness of c-MARL attack, we propose c-MARL attacks as robustness testing algorithms from multiple aspects. Experiments on \textit{SMAC} environment reveals that many state-of-the-art c-MARL algorithms are of low robustness in all aspect, pointing out the urgent need to test and enhance robustness of c-MARL algorithms.