BackWeak: Backdooring Knowledge Distillation Simply with Weak Triggers and Fine-tuning
Shanmin Wang, Dongdong Zhao

TL;DR
BackWeak introduces a simple, stealthy backdoor attack on knowledge distillation by fine-tuning with weak triggers, demonstrating high transferability and efficiency without complex surrogate models.
Contribution
It presents a novel, surrogate-free backdoor attack method that uses minimal fine-tuning with weak triggers, simplifying previous complex approaches.
Findings
High attack success rates across diverse models and datasets
BackWeak is more stealthy and efficient than prior methods
Effective transferability during standard distillation processes
Abstract
Knowledge Distillation (KD) is essential for compressing large models, yet relying on pre-trained "teacher" models downloaded from third-party repositories introduces serious security risks -- most notably backdoor attacks. Existing KD backdoor methods are typically complex and computationally intensive: they employ surrogate student models and simulated distillation to guarantee transferability, and they construct triggers in a way similar to universal adversarial perturbations (UAPs), which being not stealthy in magnitude, inherently exhibit strong adversarial behavior. This work questions whether such complexity is necessary and constructs stealthy "weak" triggers -- imperceptible perturbations that have negligible adversarial effect. We propose BackWeak, a simple, surrogate-free attack paradigm. BackWeak shows that a powerful backdoor can be implanted by simply fine-tuning a benign…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI) · Advanced Malware Detection Techniques
