TokenBinder: Text-Video Retrieval with One-to-Many Alignment Paradigm

TL;DR

TokenBinder introduces a novel one-to-many alignment paradigm for text-video retrieval, inspired by human comparative judgment, significantly improving fine-grained matching accuracy across multiple datasets.

Contribution

It proposes a two-stage framework with a Focused-view Fusion Network that dynamically aligns multiple videos, addressing limitations of one-to-one paradigms in TVR.

Findings

Outperforms state-of-the-art methods on six benchmark datasets

Demonstrates robustness and effectiveness of fine-grained alignment

Bridges intra- and inter-modality information gaps in TVR

Abstract

Text-Video Retrieval (TVR) methods typically match query-candidate pairs by aligning text and video features in coarse-grained, fine-grained, or combined (coarse-to-fine) manners. However, these frameworks predominantly employ a one(query)-to-one(candidate) alignment paradigm, which struggles to discern nuanced differences among candidates, leading to frequent mismatches. Inspired by Comparative Judgement in human cognitive science, where decisions are made by directly comparing items rather than evaluating them independently, we propose TokenBinder. This innovative two-stage TVR framework introduces a novel one-to-many coarse-to-fine alignment paradigm, imitating the human cognitive process of identifying specific items within a large collection. Our method employs a Focused-view Fusion Network with a sophisticated cross-attention mechanism, dynamically aligning and comparing features across multiple videos to capture finer nuances and contextual variations. Extensive experiments on six benchmark datasets confirm that TokenBinder substantially outperforms existing state-of-the-art methods. These results demonstrate its robustness and the effectiveness of its fine-grained alignment in bridging intra- and inter-modality information gaps in TVR tasks.