BadRes: Reveal the Backdoors through Residual Connection

TL;DR

This paper introduces BadRes, a backdoor attack exploiting residual connections in CNNs and Transformers, achieving high success rates without affecting model performance on clean data.

Contribution

The paper presents a novel backdoor attack method that leverages residual connections to bypass defenses and demonstrates its effectiveness on vision models.

Findings

BadRes achieves 97% attack success rate.

It causes no performance loss on clean data.

Residual connections are a fundamental vulnerability.

Abstract

Generally, residual connections are indispensable network components in building CNNs and Transformers for various downstream tasks in CV and VL, which encourages skip shortcuts between network blocks. However, the layer-by-layer loopback residual connections may also hurt the model's robustness by allowing unsuspecting input. In this paper, we proposed a simple yet strong backdoor attack method - BadRes, where the residual connections play as a turnstile to be deterministic on clean inputs while unpredictable on poisoned ones. We have performed empirical evaluations on four datasets with ViT and BEiT models, and the BadRes achieves 97% attack success rate while receiving zero performance degradation on clean data. Moreover, we analyze BadRes with state-of-the-art defense methods and reveal the fundamental weakness lying in residual connections.