Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examples

TL;DR

This paper introduces a provably effective black-box adversarial attack method using knowledge distillation, which iteratively trains a surrogate model to reliably generate transferable adversarial examples against neural networks.

Contribution

It is the first to provide theoretical guarantees for the success of knowledge distillation-based adversarial attacks on neural networks.

Findings

Proves finite-step success guarantee for the attack.

Demonstrates efficiency over query-based methods.

Establishes theoretical foundations for transferability.

Abstract

The vulnerability of artificial neural networks to adversarial perturbations in the black-box setting is widely studied in the literature. The majority of attack methods to construct these perturbations suffer from an impractically large number of queries required to find an adversarial example. In this work, we focus on knowledge distillation as an approach to conduct transfer-based black-box adversarial attacks and propose an iterative training of the surrogate model on an expanding dataset. This work is the first, to our knowledge, to provide provable guarantees on the success of knowledge distillation-based attack on classification neural networks: we prove that if the student model has enough learning capabilities, the attack on the teacher model is guaranteed to be found within the finite number of distillation iterations.