A Simple and Strong Baseline for Universal Targeted Attacks on Siamese Visual Tracking

TL;DR

This paper introduces a universal, video-agnostic adversarial attack method for Siamese visual trackers that requires no additional inference, effectively fooling trackers across different videos and architectures with minimal computational cost.

Contribution

The authors propose a novel universal targeted attack approach that is both data-agnostic and network-agnostic, eliminating the need for per-video optimization.

Findings

Effective fooling of Siamese trackers with universal perturbations

Perturbations generalize across different videos and models

Attack operates with minimal computational overhead

Abstract

Siamese trackers are shown to be vulnerable to adversarial attacks recently. However, the existing attack methods craft the perturbations for each video independently, which comes at a non-negligible computational cost. In this paper, we show the existence of universal perturbations that can enable the targeted attack, e.g., forcing a tracker to follow the ground-truth trajectory with specified offsets, to be video-agnostic and free from inference in a network. Specifically, we attack a tracker by adding a universal imperceptible perturbation to the template image and adding a fake target, i.e., a small universal adversarial patch, into the search images adhering to the predefined trajectory, so that the tracker outputs the location and size of the fake target instead of the real target. Our approach allows perturbing a novel video to come at no additional cost except the mere addition operations -- and not require gradient optimization or network inference. Experimental results on several datasets demonstrate that our approach can effectively fool the Siamese trackers in a targeted attack manner. We show that the proposed perturbations are not only universal across videos, but also generalize well across different trackers. Such perturbations are therefore doubly universal, both with respect to the data and the network architectures. We will make our code publicly available.