Multi-path Exploration and Feedback Adjustment for Text-to-Image Person Retrieval

TL;DR

This paper introduces MeFa, a novel multi-path framework that enhances text-to-image person retrieval by exploring intra- and inter-modal feedback for more accurate cross-modal matching.

Contribution

The paper proposes a multi-path exploration and feedback adjustment framework that improves person retrieval accuracy by leveraging intra- and inter-modal feedback mechanisms.

Findings

MeFa outperforms existing methods on three public benchmarks.

It achieves superior retrieval performance without extra data or complex models.

The framework effectively refines local and global semantic representations.

Abstract

Text-based person retrieval aims to identify the specific persons using textual descriptions as queries. Existing ad vanced methods typically depend on vision-language pre trained (VLP) models to facilitate effective cross-modal alignment. However, the inherent constraints of VLP mod-els, which include the global alignment biases and insuffi-cient self-feedback regulation, impede optimal retrieval per formance. In this paper, we propose MeFa, a Multi-Pathway Exploration, Feedback, and Adjustment framework, which deeply explores intrinsic feedback of intra and inter-modal to make targeted adjustment, thereby achieving more precise person-text associations. Specifically, we first design an intra modal reasoning pathway that generates hard negative sam ples for cross-modal data, leveraging feedback from these samples to refine intra-modal reasoning, thereby enhancing sensitivity to subtle discrepancies. Subsequently, we intro duce a cross-modal refinement pathway that utilizes both global information and intermodal feedback to refine local in formation, thus enhancing its global semantic representation. Finally, the discriminative clue correction pathway incorpo rates fine-grained features of secondary similarity as discrim inative clues to further mitigate retrieval failures caused by disparities in these features. Experimental results on three public benchmarks demonstrate that MeFa achieves superior person retrieval performance without necessitating additional data or complex structures.