Towards Generic and Controllable Attacks Against Object Detection

TL;DR

This paper introduces a generic, controllable white-box attack method called LGP that effectively fools various object detectors by focusing perturbations on foreground objects, improving transferability and reducing unnecessary perturbations.

Contribution

The paper proposes LGP, a detector-agnostic and controllable attack method that tracks proposals and optimizes heterogeneous losses, overcoming limitations of prior detector-specific and background-focused attacks.

Findings

Successfully attacked 16 state-of-the-art detectors.

Achieved promising imperceptibility and transferability.

Demonstrated effectiveness across MS-COCO and DOTA datasets.

Abstract

Existing adversarial attacks against Object Detectors (ODs) suffer from two inherent limitations. Firstly, ODs have complicated meta-structure designs, hence most advanced attacks for ODs concentrate on attacking specific detector-intrinsic structures, which makes it hard for them to work on other detectors and motivates us to design a generic attack against ODs. Secondly, most works against ODs make Adversarial Examples (AEs) by generalizing image-level attacks from classification to detection, which brings redundant computations and perturbations in semantically meaningless areas (e.g., backgrounds) and leads to an emergency for seeking controllable attacks for ODs. To this end, we propose a generic white-box attack, LGP (local perturbations with adaptively global attacks), to blind mainstream object detectors with controllable perturbations. For a detector-agnostic attack, LGP tracks high-quality proposals and optimizes three heterogeneous losses simultaneously. In this way, we can fool the crucial components of ODs with a part of their outputs without the limitations of specific structures. Regarding controllability, we establish an object-wise constraint that exploits foreground-background separation adaptively to induce the attachment of perturbations to foregrounds. Experimentally, the proposed LGP successfully attacked sixteen state-of-the-art object detectors on MS-COCO and DOTA datasets, with promising imperceptibility and transferability obtained. Codes are publicly released in https://github.com/liguopeng0923/LGP.git