Memory Backdoor Attacks on Neural Networks

TL;DR

This paper introduces a novel backdoor attack in federated learning that enables a malicious server to systematically and exactly extract private training data from client models with minimal impact on model utility.

Contribution

The authors present a new training time backdoor attack that guarantees precise data recovery and high robustness, exposing a critical privacy vulnerability in federated learning systems.

Findings

Thousands of sensitive samples can be recovered from client models.

Complete datasets can be stolen after multiple federated learning rounds.

Minimal utility drop (e.g., 3%) in models during data extraction.

Abstract

Neural networks are often trained on proprietary datasets, making them attractive attack targets. We present a novel dataset extraction method leveraging an innovative training time backdoor attack, allowing a malicious federated learning server to systematically and deterministically extract complete client training samples through a simple indexing process. Unlike prior techniques, our approach guarantees exact data recovery rather than probabilistic reconstructions or hallucinations, provides precise control over which samples are memorized and how many, and shows high capacity and robustness. Infected models output data samples when they receive a patternbased index trigger, enabling systematic extraction of meaningful patches from each clients local data without disrupting global model utility. To address small model output sizes, we extract patches and then recombined them. The attack requires only a minor modification to the training code that can easily evade detection during client-side verification. Hence, this vulnerability represents a realistic FL supply-chain threat, where a malicious server can distribute modified training code to clients and later recover private data from their updates. Evaluations across classifiers, segmentation models, and large language models demonstrate that thousands of sensitive training samples can be recovered from client models with minimal impact on task performance, and a clients entire dataset can be stolen after multiple FL rounds. For instance, a medical segmentation dataset can be extracted with only a 3 percent utility drop. These findings expose a critical privacy vulnerability in FL systems, emphasizing the need for stronger integrity and transparency in distributed training pipelines.