IoU Attack: Towards Temporally Coherent Black-Box Adversarial Attack for Visual Object Tracking

TL;DR

This paper introduces a novel decision-based black-box adversarial attack called IoU attack, which degrades the performance of deep visual object trackers by sequentially generating perturbations based on IoU scores, without requiring model knowledge.

Contribution

The paper proposes the first IoU-based black-box attack method for visual tracking that effectively reduces tracker accuracy without model access.

Findings

Effective degradation of tracker accuracy demonstrated

Applicable to various types of deep trackers

Outperforms existing black-box attack methods

Abstract

Adversarial attack arises due to the vulnerability of deep neural networks to perceive input samples injected with imperceptible perturbations. Recently, adversarial attack has been applied to visual object tracking to evaluate the robustness of deep trackers. Assuming that the model structures of deep trackers are known, a variety of white-box attack approaches to visual tracking have demonstrated promising results. However, the model knowledge about deep trackers is usually unavailable in real applications. In this paper, we propose a decision-based black-box attack method for visual object tracking. In contrast to existing black-box adversarial attack methods that deal with static images for image classification, we propose IoU attack that sequentially generates perturbations based on the predicted IoU scores from both current and historical frames. By decreasing the IoU scores, the proposed attack method degrades the accuracy of temporal coherent bounding boxes (i.e., object motions) accordingly. In addition, we transfer the learned perturbations to the next few frames to initialize temporal motion attack. We validate the proposed IoU attack on state-of-the-art deep trackers (i.e., detection based, correlation filter based, and long-term trackers). Extensive experiments on the benchmark datasets indicate the effectiveness of the proposed IoU attack method. The source code is available at https://github.com/VISION-SJTU/IoUattack.