DAPL: Integration of Positive and Negative Descriptions in Text-Based Person Search

TL;DR

This paper introduces DAPL, a novel framework for text-based person search that integrates positive and negative descriptions, employing dual contrastive and matching techniques to improve accuracy and robustness.

Contribution

DAPL is the first to incorporate negative descriptions in TBPS using dual contrastive and token-wise similarity losses for better interpretative accuracy.

Findings

DAPL outperforms existing methods in TBPS accuracy.

The DTS loss improves fine-grained visual-text alignment.

Inclusion of negative descriptions reduces false positives.

Abstract

Text-based person search (TBPS) aims to retrieve specific images of individuals from large datasets using textual descriptions. Existing TBPS methods focus primarily on identifying explicit positive attributes, often neglecting the critical role of negative descriptions. This oversight can lead to false positives, where images that should be excluded based on negative descriptions are incorrectly included, due to partial alignment with the positive criteria. To address this limitation, we propose the Dual Attribute Prompt Learning (DAPL) framework, which incorporates both positive and negative descriptions to improve the interpretative accuracy of vision-language models in TBPS tasks. DAPL combines Dual Image-Attribute Contrastive (DIAC) learning with Sensitive Image-Attribute Matching (SIAM) learning to enhance the detection of previously unseen attributes. Furthermore, to achieve a balance between coarse and fine-grained alignment of visual and textual embeddings, we introduce the Dynamic Token-wise Similarity (DTS) loss. This loss function refines the representation of both matching and non-matching descriptions at the token level, providing more precise and adaptable similarity assessments, and ultimately improving the accuracy of the matching process. Empirical results demonstrate that DAPL outperforms state-of-the-art methods, enhancing both precision and robustness in TBPS tasks.