An anomaly detection approach for backdoored neural networks: face recognition as a case study

TL;DR

This paper presents a new anomaly detection method to identify backdoored neural networks, demonstrated on face recognition, achieving perfect detection scores without assumptions on backdoor specifics.

Contribution

The paper introduces a novel backdoor detection approach based on anomaly detection that works without prior knowledge of backdoor characteristics.

Findings

Achieved perfect detection scores on a new backdoored network dataset.

Effective across various triggers, locations, and identity pairs.

Does not require assumptions about backdoor nature or setup.

Abstract

Backdoor attacks allow an attacker to embed functionality jeopardizing proper behavior of any algorithm, machine learning or not. This hidden functionality can remain inactive for normal use of the algorithm until activated by the attacker. Given how stealthy backdoor attacks are, consequences of these backdoors could be disastrous if such networks were to be deployed for applications as critical as border or access control. In this paper, we propose a novel backdoored network detection method based on the principle of anomaly detection, involving access to the clean part of the training data and the trained network. We highlight its promising potential when considering various triggers, locations and identity pairs, without the need to make any assumptions on the nature of the backdoor and its setup. We test our method on a novel dataset of backdoored networks and report detectability results with perfect scores.