Progressive Feature Mining and External Knowledge-Assisted Text-Pedestrian Image Retrieval

TL;DR

This paper introduces a novel method for text-pedestrian image retrieval that employs progressive feature mining and external knowledge to enhance discriminative features and reduce modality and textual diversity issues, achieving superior results.

Contribution

The paper proposes a progressive feature mining approach combined with external knowledge-assisted feature purification to improve cross-modal pedestrian retrieval performance.

Findings

Outperforms existing methods on three challenging datasets.

Surpasses large-scale model-based methods on large-scale datasets.

Effectively reduces modality discrepancy and textual diversity impacts.

Abstract

Text-Pedestrian Image Retrieval aims to use the text describing pedestrian appearance to retrieve the corresponding pedestrian image. This task involves not only modality discrepancy, but also the challenge of the textual diversity of pedestrians with the same identity. At present, although existing research progress has been made in text-pedestrian image retrieval, these methods do not comprehensively consider the above-mentioned problems. Considering these, this paper proposes a progressive feature mining and external knowledge-assisted feature purification method. Specifically, we use a progressive mining mode to enable the model to mine discriminative features from neglected information, thereby avoiding the loss of discriminative information and improving the expression ability of features. In addition, to further reduce the negative impact of modal discrepancy and text diversity on cross-modal matching, we propose to use other sample knowledge of the same modality, i.e., external knowledge to enhance identity-consistent features and weaken identity-inconsistent features. This process purifies features and alleviates the interference caused by textual diversity and negative sample correlation features of the same modal. Extensive experiments on three challenging datasets demonstrate the effectiveness and superiority of the proposed method, and the retrieval performance even surpasses that of the large-scale model-based method on large-scale datasets.