Dynamic Uncertainty Learning with Noisy Correspondence for Text-Based Person Search

TL;DR

This paper introduces a novel framework, DURA, that models noise uncertainty and adapts to noisy data in text-based person search, significantly improving robustness and retrieval accuracy.

Contribution

The paper proposes the DURA framework with KFS and DSH-Loss to effectively handle noisy correspondence in text-image person search, a novel approach in this domain.

Findings

Enhanced noise resistance in retrieval performance

Improved accuracy in both low- and high-noise scenarios

Effective modeling of cross-modal similarity as a Dirichlet distribution

Abstract

Text-to-image person search aims to identify an individual based on a text description. To reduce data collection costs, large-scale text-image datasets are created from co-occurrence pairs found online. However, this can introduce noise, particularly mismatched pairs, which degrade retrieval performance. Existing methods often focus on negative samples, which amplify this noise. To address these issues, we propose the Dynamic Uncertainty and Relational Alignment (DURA) framework, which includes the Key Feature Selector (KFS) and a new loss function, Dynamic Softmax Hinge Loss (DSH-Loss). KFS captures and models noise uncertainty, improving retrieval reliability. The bidirectional evidence from cross-modal similarity is modeled as a Dirichlet distribution, enhancing adaptability to noisy data. DSH adjusts the difficulty of negative samples to improve robustness in noisy environments. Our experiments on three datasets show that the method offers strong noise resistance and improves retrieval performance in both low- and high-noise scenarios.