ImpNet: Imperceptible and blackbox-undetectable backdoors in compiled neural networks

TL;DR

This paper introduces ImpNet, a novel class of imperceptible, blackbox-undetectable backdoors in neural networks that are inserted during compilation, challenging existing defense mechanisms and emphasizing the need for provenance assurance throughout the ML pipeline.

Contribution

The work presents ImpNet, a new class of backdoors that are inserted during compilation and are undetectable by current defenses, highlighting vulnerabilities in the ML development process.

Findings

Backdoors can be inserted during model compilation, bypassing data and training safeguards.

ImpNet backdoors are imperceptible and undetectable during training or data inspection.

Detecting and removing such backdoors is only reliable at the insertion stage.

Abstract

Early backdoor attacks against machine learning set off an arms race in attack and defence development. Defences have since appeared demonstrating some ability to detect backdoors in models or even remove them. These defences work by inspecting the training data, the model, or the integrity of the training procedure. In this work, we show that backdoors can be added during compilation, circumventing any safeguards in the data preparation and model training stages. The attacker can not only insert existing weight-based backdoors during compilation, but also a new class of weight-independent backdoors, such as ImpNet. These backdoors are impossible to detect during the training or data preparation processes, because they are not yet present. Next, we demonstrate that some backdoors, including ImpNet, can only be reliably detected at the stage where they are inserted and removing them anywhere else presents a significant challenge. We conclude that ML model security requires assurance of provenance along the entire technical pipeline, including the data, model architecture, compiler, and hardware specification.