T2I-VeRW: Part-level Fine-grained Perception for Text-to-Image Vehicle Retrieval

TL;DR

This paper introduces PFCVR, a novel part-level fine-grained cross-modal vehicle retrieval model for text-to-image re-identification, along with a new large-scale dataset T2I-VeRW.

Contribution

The paper proposes a new model with local part-level alignment and a bi-directional mask recovery module, and constructs a large-scale dataset for text-to-image vehicle re-identification.

Findings

PFCVR achieves 29.2% Rank-1 accuracy on T2I-VeRI, surpassing previous methods.

On T2I-VeRW, PFCVR attains 55.2% Rank-1 accuracy, outperforming recent state-of-the-art models.

Abstract

Vehicle Re-identification (Re-ID) aims to retrieve the most similar image to a given query from images captured by non-overlapping cameras. Extending vehicle Re-ID from image-only queries to text-based queries enables retrieval in real-world scenarios where only a witness description of the target vehicle is available. In this paper, we propose PFCVR, a Part-level Fine-grained Cross-modal Vehicle Retrieval model for text-to-image vehicle re-identification. PFCVR constructs locally paired images and texts at the part level and introduces learnable part-query tokens that aggregate both part-specific and full-sentence context before aligning with visual part features. On top of this explicit local alignment, a bi-directional mask recovery module lets each modality reconstruct its masked content under the guidance of the other, implicitly bridging local correspondences into global feature alignment. Furthermore, we construct a new large-scale dataset called T2I-VeRW, which contains 14,668 images covering 1,796 vehicle identities with fine-grained part-level annotations. Experimental results on the T2I-VeRI dataset show that PFCVR achieves 29.2\% Rank-1 accuracy, improving over the best competing method by +3.7\% percentage points. On the newly proposed T2I-VeRW benchmark, PFCVR achieves 55.2\% Rank-1 accuracy, outperforming a comprehensive set of recent state-of-the-art methods. Source code will be released on https://github.com/Event-AHU/Neuromorphic_ReID