Thinking Two Moves Ahead: Anticipating Other Users Improves Backdoor Attacks in Federated Learning

TL;DR

This paper introduces a novel backdoor attack in federated learning that anticipates future client behaviors, enabling effective and persistent backdoors even with limited attacker participation across multiple tasks.

Contribution

The paper presents a new attack method that considers the entire federated learning process, improving backdoor effectiveness and persistence compared to prior approaches.

Findings

Effective backdoor attacks with limited participation

Persists across multiple rounds and tasks

Applicable to various machine learning tasks

Abstract

Federated learning is particularly susceptible to model poisoning and backdoor attacks because individual users have direct control over the training data and model updates. At the same time, the attack power of an individual user is limited because their updates are quickly drowned out by those of many other users. Existing attacks do not account for future behaviors of other users, and thus require many sequential updates and their effects are quickly erased. We propose an attack that anticipates and accounts for the entire federated learning pipeline, including behaviors of other clients, and ensures that backdoors are effective quickly and persist even after multiple rounds of community updates. We show that this new attack is effective in realistic scenarios where the attacker only contributes to a small fraction of randomly sampled rounds and demonstrate this attack on image classification, next-word prediction, and sentiment analysis.