Generative Adversarial Patches for Physical Attacks on Cross-Modal Pedestrian Re-Identification

TL;DR

This paper introduces a novel physical adversarial attack called Edge-Attack on cross-modal pedestrian re-identification systems, using edge-based features and generative models to create realistic patches that significantly impair model performance.

Contribution

It presents the first physical attack targeting VI-ReID models, leveraging edge features and ViTGAN to generate effective adversarial patches in a black-box, self-supervised manner.

Findings

Edge-Attack significantly reduces VI-ReID accuracy on benchmark datasets.

The method is effective in real-world scenarios and across various models.

The attack exploits edge features to deceive deep-level implicit representations.

Abstract

Visible-infrared pedestrian Re-identification (VI-ReID) aims to match pedestrian images captured by infrared cameras and visible cameras. However, VI-ReID, like other traditional cross-modal image matching tasks, poses significant challenges due to its human-centered nature. This is evidenced by the shortcomings of existing methods, which struggle to extract common features across modalities, while losing valuable information when bridging the gap between them in the implicit feature space, potentially compromising security. To address this vulnerability, this paper introduces the first physical adversarial attack against VI-ReID models. Our method, termed Edge-Attack, specifically tests the models' ability to leverage deep-level implicit features by focusing on edge information, the most salient explicit feature differentiating individuals across modalities. Edge-Attack utilizes a novel two-step approach. First, a multi-level edge feature extractor is trained in a self-supervised manner to capture discriminative edge representations for each individual. Second, a generative model based on Vision Transformer Generative Adversarial Networks (ViTGAN) is employed to generate adversarial patches conditioned on the extracted edge features. By applying these patches to pedestrian clothing, we create realistic, physically-realizable adversarial samples. This black-box, self-supervised approach ensures the generalizability of our attack against various VI-ReID models. Extensive experiments on SYSU-MM01 and RegDB datasets, including real-world deployments, demonstrate the effectiveness of Edge- Attack in significantly degrading the performance of state-of-the-art VI-ReID methods.