SSMI: How to Make Objects of Interest Disappear without Accessing Object Detectors?

TL;DR

This paper introduces SSMI, a black-box adversarial attack method that effectively makes objects disappear from detection without needing access to object detectors, outperforming existing methods in efficiency and success rate.

Contribution

The paper presents a novel black-box attack scheme using semantic segmentation and model inversion to make objects disappear without detector access, improving efficiency and success rate.

Findings

Achieves up to 16% increase in new/disappearing labels.

Reduces mAP metrics for object detection by up to 36%.

Generates adversarial examples that evade human perception.

Abstract

Most black-box adversarial attack schemes for object detectors mainly face two shortcomings: requiring access to the target model and generating inefficient adversarial examples (failing to make objects disappear in large numbers). To overcome these shortcomings, we propose a black-box adversarial attack scheme based on semantic segmentation and model inversion (SSMI). We first locate the position of the target object using semantic segmentation techniques. Next, we design a neighborhood background pixel replacement to replace the target region pixels with background pixels to ensure that the pixel modifications are not easily detected by human vision. Finally, we reconstruct a machine-recognizable example and use the mask matrix to select pixels in the reconstructed example to modify the benign image to generate an adversarial example. Detailed experimental results show that SSMI can generate efficient adversarial examples to evade human-eye perception and make objects of interest disappear. And more importantly, SSMI outperforms existing same kinds of attacks. The maximum increase in new and disappearing labels is 16%, and the maximum decrease in mAP metrics for object detection is 36%.