Widen The Backdoor To Let More Attackers In

TL;DR

This paper explores multi-agent backdoor attacks in collaborative learning, revealing that more attackers reduce individual success rates and proposing defenses by increasing attackers artificially and removing attacker data during inference.

Contribution

It uncovers the backfiring effect in multi-agent backdoor attacks and proposes two defenses leveraging this phenomenon to improve model robustness.

Findings

Increasing attackers decreases attack success rate

Artificially augmenting attackers reduces attack effectiveness

Removing attacker data during inference enhances robustness

Abstract

As collaborative learning and the outsourcing of data collection become more common, malicious actors (or agents) which attempt to manipulate the learning process face an additional obstacle as they compete with each other. In backdoor attacks, where an adversary attempts to poison a model by introducing malicious samples into the training data, adversaries have to consider that the presence of additional backdoor attackers may hamper the success of their own backdoor. In this paper, we investigate the scenario of a multi-agent backdoor attack, where multiple non-colluding attackers craft and insert triggered samples in a shared dataset which is used by a model (a defender) to learn a task. We discover a clear backfiring phenomenon: increasing the number of attackers shrinks each attacker's attack success rate (ASR). We then exploit this phenomenon to minimize the collective ASR of attackers and maximize defender's robustness accuracy by (i) artificially augmenting the number of attackers, and (ii) indexing to remove the attacker's sub-dataset from the model for inference, hence proposing 2 defenses.