Prompt-driven Transferable Adversarial Attack on Person Re-Identification with Attribute-aware Textual Inversion

TL;DR

This paper introduces AP-Attack, a novel prompt-driven adversarial attack method that disrupts attribute-specific features in person re-identification models, significantly improving transferability across models and datasets.

Contribution

The paper proposes Attribute-aware Prompt Attack (AP-Attack), leveraging textual inversion and attribute-specific disruptions to enhance adversarial transferability in person re-id.

Findings

Achieves 22.9% improvement in mean Drop Rate over previous methods.

Effectively disrupts fine-grained semantic features of pedestrian images.

Demonstrates superior transferability across models and datasets.

Abstract

Person re-identification (re-id) models are vital in security surveillance systems, requiring transferable adversarial attacks to explore the vulnerabilities of them. Recently, vision-language models (VLM) based attacks have shown superior transferability by attacking generalized image and textual features of VLM, but they lack comprehensive feature disruption due to the overemphasis on discriminative semantics in integral representation. In this paper, we introduce the Attribute-aware Prompt Attack (AP-Attack), a novel method that leverages VLM's image-text alignment capability to explicitly disrupt fine-grained semantic features of pedestrian images by destroying attribute-specific textual embeddings. To obtain personalized textual descriptions for individual attributes, textual inversion networks are designed to map pedestrian images to pseudo tokens that represent semantic embeddings, trained in the contrastive learning manner with images and a predefined prompt template that explicitly describes the pedestrian attributes. Inverted benign and adversarial fine-grained textual semantics facilitate attacker in effectively conducting thorough disruptions, enhancing the transferability of adversarial examples. Extensive experiments show that AP-Attack achieves state-of-the-art transferability, significantly outperforming previous methods by 22.9% on mean Drop Rate in cross-model&dataset attack scenarios.