Modality Unified Attack for Omni-Modality Person Re-Identification

TL;DR

This paper introduces a novel attack method that generates adversarial examples capable of fooling omni-modality person re-identification models across single, cross, and multi-modality scenarios, revealing vulnerabilities in multi-modal surveillance systems.

Contribution

The paper proposes a Modality Unified Attack method that trains modality-specific adversarial generators to effectively attack various omni-modality re-id models, a novel approach in this domain.

Findings

Achieves up to 62.7% mean mAP Drop Rate on targeted models.

Effectively attacks multi-modality re-id systems across different modalities.

Demonstrates significant vulnerability of omni-modality models to the proposed attack.

Abstract

Deep learning based person re-identification (re-id) models have been widely employed in surveillance systems. Recent studies have demonstrated that black-box single-modality and cross-modality re-id models are vulnerable to adversarial examples (AEs), leaving the robustness of multi-modality re-id models unexplored. Due to the lack of knowledge about the specific type of model deployed in the target black-box surveillance system, we aim to generate modality unified AEs for omni-modality (single-, cross- and multi-modality) re-id models. Specifically, we propose a novel Modality Unified Attack method to train modality-specific adversarial generators to generate AEs that effectively attack different omni-modality models. A multi-modality model is adopted as the surrogate model, wherein the features of each modality are perturbed by metric disruption loss before fusion. To collapse the common features of omni-modality models, Cross Modality Simulated Disruption approach is introduced to mimic the cross-modality feature embeddings by intentionally feeding images to non-corresponding modality-specific subnetworks of the surrogate model. Moreover, Multi Modality Collaborative Disruption strategy is devised to facilitate the attacker to comprehensively corrupt the informative content of person images by leveraging a multi modality feature collaborative metric disruption loss. Extensive experiments show that our MUA method can effectively attack the omni-modality re-id models, achieving 55.9%, 24.4%, 49.0% and 62.7% mean mAP Drop Rate, respectively.