Relevance Attack on Detectors

TL;DR

This paper introduces a novel relevance-based attack method called RAD that significantly improves transferability of adversarial attacks on detectors, enabling the creation of a new adversarial dataset for robustness evaluation.

Contribution

It is the first to use relevance maps from interpreters as a common weakness to enhance attack transferability on detectors, achieving state-of-the-art results.

Findings

RAD exceeds existing transferability by over 20%

Detection mAPs are more than halved across 8 architectures

Created the first adversarial dataset for object detection and segmentation

Abstract

This paper focuses on high-transferable adversarial attacks on detectors, which are hard to attack in a black-box manner, because of their multiple-output characteristics and the diversity across architectures. To pursue a high attack transferability, one plausible way is to find a common property across detectors, which facilitates the discovery of common weaknesses. We are the first to suggest that the relevance map from interpreters for detectors is such a property. Based on it, we design a Relevance Attack on Detectors (RAD), which achieves a state-of-the-art transferability, exceeding existing results by above 20%. On MS COCO, the detection mAPs for all 8 black-box architectures are more than halved and the segmentation mAPs are also significantly influenced. Given the great transferability of RAD, we generate the first adversarial dataset for object detection and instance segmentation, i.e., Adversarial Objects in COntext (AOCO), which helps to quickly evaluate and improve the robustness of detectors.