On the Importance of Backbone to the Adversarial Robustness of Object Detectors

TL;DR

This paper demonstrates that using adversarially pre-trained backbones significantly improves the robustness of object detectors against adversarial attacks, surpassing previous methods without changing detector structures.

Contribution

It introduces a simple recipe for fast adversarial fine-tuning with pre-trained backbones, leading to state-of-the-art robustness in object detection.

Findings

Adversarially pre-trained backbones are crucial for robustness.

The proposed fine-tuning recipe outperforms previous methods.

Modern detector designs can be optimized for better robustness.

Abstract

Object detection is a critical component of various security-sensitive applications, such as autonomous driving and video surveillance. However, existing object detectors are vulnerable to adversarial attacks, which poses a significant challenge to their reliability and security. Through experiments, first, we found that existing works on improving the adversarial robustness of object detectors give a false sense of security. Second, we found that adversarially pre-trained backbone networks were essential for enhancing the adversarial robustness of object detectors. We then proposed a simple yet effective recipe for fast adversarial fine-tuning on object detectors with adversarially pre-trained backbones. Without any modifications to the structure of object detectors, our recipe achieved significantly better adversarial robustness than previous works. Finally, we explored the potential of different modern object detector designs for improving adversarial robustness with our recipe and demonstrated interesting findings, which inspired us to design state-of-the-art (SOTA) robust detectors. Our empirical results set a new milestone for adversarially robust object detection. Code and trained checkpoints are available at https://github.com/thu-ml/oddefense.