Architectural Backdoors in Neural Networks

TL;DR

This paper introduces a novel class of backdoor attacks in neural networks that embed malicious functionalities directly into model architectures, posing a significant threat due to their persistence even after retraining.

Contribution

It formalizes the concept of architectural backdoors, demonstrating their feasibility, threat level, and potential defenses across various computer vision benchmarks.

Findings

Architectural backdoors can survive complete retraining.

They are easy to implement via open-source code.

Vulnerable across different training settings.

Abstract

Machine learning is vulnerable to adversarial manipulation. Previous literature has demonstrated that at the training stage attackers can manipulate data and data sampling procedures to control model behaviour. A common attack goal is to plant backdoors i.e. force the victim model to learn to recognise a trigger known only by the adversary. In this paper, we introduce a new class of backdoor attacks that hide inside model architectures i.e. in the inductive bias of the functions used to train. These backdoors are simple to implement, for instance by publishing open-source code for a backdoored model architecture that others will reuse unknowingly. We demonstrate that model architectural backdoors represent a real threat and, unlike other approaches, can survive a complete re-training from scratch. We formalise the main construction principles behind architectural backdoors, such as a link between the input and the output, and describe some possible protections against them. We evaluate our attacks on computer vision benchmarks of different scales and demonstrate the underlying vulnerability is pervasive in a variety of training settings.