T-SEA: Transfer-based Self-Ensemble Attack on Object Detection

TL;DR

This paper introduces T-SEA, a novel single-model transfer-based black-box attack on object detection that employs self-ensemble strategies to enhance transferability and effectiveness across multiple detectors without relying on model ensembling.

Contribution

The paper proposes a new self-ensemble attack framework for object detection that improves transferability using only one model, reducing resource needs compared to existing multi-model approaches.

Findings

Significantly improves black-box transferability of adversarial patches.

Enhances white-box attack performance.

Compatible with classical attack methods like PGD and MIM.

Abstract

Compared to query-based black-box attacks, transfer-based black-box attacks do not require any information of the attacked models, which ensures their secrecy. However, most existing transfer-based approaches rely on ensembling multiple models to boost the attack transferability, which is time- and resource-intensive, not to mention the difficulty of obtaining diverse models on the same task. To address this limitation, in this work, we focus on the single-model transfer-based black-box attack on object detection, utilizing only one model to achieve a high-transferability adversarial attack on multiple black-box detectors. Specifically, we first make observations on the patch optimization process of the existing method and propose an enhanced attack framework by slightly adjusting its training strategies. Then, we analogize patch optimization with regular model optimization, proposing a series of self-ensemble approaches on the input data, the attacked model, and the adversarial patch to efficiently make use of the limited information and prevent the patch from overfitting. The experimental results show that the proposed framework can be applied with multiple classical base attack methods (e.g., PGD and MIM) to greatly improve the black-box transferability of the well-optimized patch on multiple mainstream detectors, meanwhile boosting white-box performance. Our code is available at https://github.com/VDIGPKU/T-SEA.