Adversarial Attacks On Multi-Agent Communication

TL;DR

This paper investigates the robustness of multi-agent systems against adversarial communication attacks, revealing vulnerabilities and challenges in defending learned representations shared among autonomous agents.

Contribution

It introduces a novel multi-agent attack framework targeting shared neural representations and analyzes robustness, highlighting the difficulty of black-box transfer attacks in this setting.

Findings

Adversarial messages can significantly degrade system performance.

Increasing benign agents weakens the impact of adversarial messages.

Black-box transfer attacks are more challenging due to distribution alignment requirements.

Abstract

Growing at a fast pace, modern autonomous systems will soon be deployed at scale, opening up the possibility for cooperative multi-agent systems. Sharing information and distributing workloads allow autonomous agents to better perform tasks and increase computation efficiency. However, shared information can be modified to execute adversarial attacks on deep learning models that are widely employed in modern systems. Thus, we aim to study the robustness of such systems and focus on exploring adversarial attacks in a novel multi-agent setting where communication is done through sharing learned intermediate representations of neural networks. We observe that an indistinguishable adversarial message can severely degrade performance, but becomes weaker as the number of benign agents increases. Furthermore, we show that black-box transfer attacks are more difficult in this setting when compared to directly perturbing the inputs, as it is necessary to align the distribution of learned representations with domain adaptation. Our work studies robustness at the neural network level to contribute an additional layer of fault tolerance to modern security protocols for more secure multi-agent systems.