Towards Effective and Efficient Adversarial Defense with Diffusion Models for Robust Visual Tracking

TL;DR

This paper introduces DiffDf, a diffusion model-based adversarial defense for visual tracking that enhances robustness against attacks while maintaining real-time performance.

Contribution

It presents the first diffusion model-based adversarial defense method for visual tracking, combining multi-scale denoising with multiple loss functions for improved robustness.

Findings

Significantly improves tracking robustness against adversarial attacks.

Achieves real-time inference at over 30 FPS.

Demonstrates strong generalization across different tracker architectures.

Abstract

Although deep learning-based visual tracking methods have made significant progress, they exhibit vulnerabilities when facing carefully designed adversarial attacks, which can lead to a sharp decline in tracking performance. To address this issue, this paper proposes for the first time a novel adversarial defense method based on denoise diffusion probabilistic models, termed DiffDf, aimed at effectively improving the robustness of existing visual tracking methods against adversarial attacks. DiffDf establishes a multi-scale defense mechanism by combining pixel-level reconstruction loss, semantic consistency loss, and structural similarity loss, effectively suppressing adversarial perturbations through a gradual denoising process. Extensive experimental results on several mainstream datasets show that the DiffDf method demonstrates excellent generalization performance for trackers with different architectures, significantly improving various evaluation metrics while achieving real-time inference speeds of over 30 FPS, showcasing outstanding defense performance and efficiency. Codes are available at https://github.com/pgao-lab/DiffDf.