Mitigating Data Exfiltration Attacks through Layer-Wise Learning Rate Decay Fine-Tuning

TL;DR

This paper introduces a simple fine-tuning method with layer-wise learning rate decay to prevent data exfiltration from sensitive medical models, maintaining utility while disrupting privacy attacks.

Contribution

The authors propose a novel fine-tuning approach that effectively mitigates data exfiltration attacks without compromising model performance.

Findings

Effectively disrupts state-of-the-art exfiltration attacks

Maintains high utility task performance

Outperforms prior defense methods

Abstract

Data lakes enable the training of powerful machine learning models on sensitive, high-value medical datasets, but also introduce serious privacy risks due to potential leakage of protected health information. Recent studies show adversaries can exfiltrate training data by embedding latent representations into model parameters or inducing memorization via multi-task learning. These attacks disguise themselves as benign utility models while enabling reconstruction of high-fidelity medical images, posing severe privacy threats with legal and ethical implications. In this work, we propose a simple yet effective mitigation strategy that perturbs model parameters at export time through fine-tuning with a decaying layer-wise learning rate to corrupt embedded data without degrading task performance. Evaluations on DermaMNIST, ChestMNIST, and MIMIC-CXR show that our approach maintains utility task performance, effectively disrupts state-of-the-art exfiltration attacks, outperforms prior defenses, and renders exfiltrated data unusable for training. Ablations and discussions on adaptive attacks highlight challenges and future directions. Our findings offer a practical defense against data leakage in data lake-trained models and centralized federated learning.