SCING:Towards More Efficient and Robust Person Re-Identification through Selective Cross-modal Prompt Tuning

TL;DR

SCING introduces a lightweight, cross-modal prompt tuning framework that improves person re-identification by enhancing alignment and robustness while reducing computational costs, outperforming existing methods on multiple benchmarks.

Contribution

The paper proposes SCING, a novel framework with SVIP and PDCA modules, for efficient and robust person ReID using cross-modal prompt tuning without heavy adapters.

Findings

Achieves state-of-the-art performance on multiple ReID benchmarks.

Reduces computational overhead compared to existing adapter-based methods.

Enhances robustness against real-world image perturbations.

Abstract

Recent advancements in adapting vision-language pre-training models like CLIP for person re-identification (ReID) tasks often rely on complex adapter design or modality-specific tuning while neglecting cross-modal interaction, leading to high computational costs or suboptimal alignment. To address these limitations, we propose a simple yet effective framework named Selective Cross-modal Prompt Tuning (SCING) that enhances cross-modal alignment and robustness against real-world perturbations. Our method introduces two key innovations: Firstly, we proposed Selective Visual Prompt Fusion (SVIP), a lightweight module that dynamically injects discriminative visual features into text prompts via a cross-modal gating mechanism. Moreover, the proposed Perturbation-Driven Consistency Alignment (PDCA) is a dual-path training strategy that enforces invariant feature alignment under random image perturbations by regularizing consistency between original and augmented cross-modal embeddings. Extensive experiments are conducted on several popular benchmarks covering Market1501, DukeMTMC-ReID, Occluded-Duke, Occluded-REID, and P-DukeMTMC, which demonstrate the impressive performance of the proposed method. Notably, our framework eliminates heavy adapters while maintaining efficient inference, achieving an optimal trade-off between performance and computational overhead. The code will be released upon acceptance.