Adversarial Attack on Black-Box Multi-Agent by Adaptive Perturbation

TL;DR

This paper introduces AdapAM, a novel black-box attack framework for multi-agent systems that balances effectiveness and stealthiness using adaptive victim selection and proxy-based perturbations.

Contribution

AdapAM is the first framework to combine adaptive victim selection with proxy-based perturbation for stealthy black-box attacks on multi-agent systems.

Findings

AdapAM outperforms four baselines across eight environments.

Perturbations generated by AdapAM are less noisy and more stealthy.

AdapAM achieves the best attack performance at various perturbation rates.

Abstract

Evaluating security and reliability for multi-agent systems (MAS) is urgent as they become increasingly prevalent in various applications. As an evaluation technique, existing adversarial attack frameworks face certain limitations, e.g., impracticality due to the requirement of white-box information or high control authority, and a lack of stealthiness or effectiveness as they often target all agents or specific fixed agents. To address these issues, we propose AdapAM, a novel framework for adversarial attacks on black-box MAS. AdapAM incorporates two key components: (1) Adaptive Selection Policy simultaneously selects the victim and determines the anticipated malicious action (the action would lead to the worst impact on MAS), balancing effectiveness and stealthiness. (2) Proxy-based Perturbation to Induce Malicious Action utilizes generative adversarial imitation learning to approximate the target MAS, allowing AdapAM to generate perturbed observations using white-box information and thus induce victims to execute malicious action in black-box settings. We evaluate AdapAM across eight multi-agent environments and compare it with four state-of-the-art and commonly-used baselines. Results demonstrate that AdapAM achieves the best attack performance in different perturbation rates. Besides, AdapAM-generated perturbations are the least noisy and hardest to detect, emphasizing the stealthiness.