BDefects4NN: A Backdoor Defect Database for Controlled Localization Studies in Neural Networks

TL;DR

This paper introduces BDefects4NN, a comprehensive database of backdoor-defected neural networks, enabling controlled localization studies to improve detection and repair of malicious model defects in critical AI systems.

Contribution

It presents the first backdoor defect database with labeled defects at neuron granularity, facilitating research on defect localization and mitigation in neural networks.

Findings

Limited effectiveness of existing fault localization criteria for backdoor defects

Backdoor models pose significant threats in autonomous driving and LLMs

Current defect localization techniques need improvement

Abstract

Pre-trained large deep learning models are now serving as the dominant component for downstream middleware users and have revolutionized the learning paradigm, replacing the traditional approach of training from scratch locally. To reduce development costs, developers often integrate third-party pre-trained deep neural networks (DNNs) into their intelligent software systems. However, utilizing untrusted DNNs presents significant security risks, as these models may contain intentional backdoor defects resulting from the black-box training process. These backdoor defects can be activated by hidden triggers, allowing attackers to maliciously control the model and compromise the overall reliability of the intelligent software. To ensure the safe adoption of DNNs in critical software systems, it is crucial to establish a backdoor defect database for localization studies. This paper addresses this research gap by introducing BDefects4NN, the first backdoor defect database, which provides labeled backdoor-defected DNNs at the neuron granularity and enables controlled localization studies of defect root causes. In BDefects4NN, we define three defect injection rules and employ four representative backdoor attacks across four popular network architectures and three widely adopted datasets, yielding a comprehensive database of 1,654 backdoor-defected DNNs with four defect quantities and varying infected neurons. Based on BDefects4NN, we conduct extensive experiments on evaluating six fault localization criteria and two defect repair techniques, which show limited effectiveness for backdoor defects. Additionally, we investigate backdoor-defected models in practical scenarios, specifically in lane detection for autonomous driving and large language models (LLMs), revealing potential threats and highlighting current limitations in precise defect localization.