Purification Of Contaminated Convolutional Neural Networks Via Robust Recovery: An Approach with Theoretical Guarantee in One-Hidden-Layer Case

TL;DR

This paper introduces a theoretically guaranteed method for robustly recovering and purifying contaminated one-hidden-layer CNNs, effectively removing noise and defending against backdoor attacks, with promising extensions to deeper networks.

Contribution

It provides the first exact recovery guarantee for contaminated CNNs with ReLU activation in the one-hidden-layer case, advancing robustness and security in neural network models.

Findings

Exact recovery of CNN weights and biases under mild assumptions

Effective noise removal demonstrated in synthetic and real settings

Potential extension to multi-layer CNNs and backdoor defense

Abstract

Convolutional neural networks (CNNs), one of the key architectures of deep learning models, have achieved superior performance on many machine learning tasks such as image classification, video recognition, and power systems. Despite their success, CNNs can be easily contaminated by natural noises and artificially injected noises such as backdoor attacks. In this paper, we propose a robust recovery method to remove the noise from the potentially contaminated CNNs and provide an exact recovery guarantee on one-hidden-layer non-overlapping CNNs with the rectified linear unit (ReLU) activation function. Our theoretical results show that both CNNs' weights and biases can be exactly recovered under the overparameterization setting with some mild assumptions. The experimental results demonstrate the correctness of the proofs and the effectiveness of the method in both the synthetic environment and the practical neural network setting. Our results also indicate that the proposed method can be extended to multiple-layer CNNs and potentially serve as a defense strategy against backdoor attacks.