HiT: Hierarchical Transformer with Momentum Contrast for Video-Text Retrieval

TL;DR

This paper introduces HiT, a hierarchical transformer with momentum contrast for improved video-text retrieval, leveraging multi-level contrastive learning and large-scale negative sample interactions to enhance retrieval accuracy.

Contribution

The paper proposes a novel Hierarchical Transformer with momentum contrast, enabling multi-view and comprehensive retrieval with large-scale negative sample interactions.

Findings

Outperforms existing methods on major benchmarks

Effective multi-level contrastive learning improves retrieval accuracy

Momentum contrast enables large-scale negative sample interaction

Abstract

Video-Text Retrieval has been a hot research topic with the growth of multimedia data on the internet. Transformer for video-text learning has attracted increasing attention due to its promising performance. However, existing cross-modal transformer approaches typically suffer from two major limitations: 1) Exploitation of the transformer architecture where different layers have different feature characteristics is limited; 2) End-to-end training mechanism limits negative sample interactions in a mini-batch. In this paper, we propose a novel approach named Hierarchical Transformer (HiT) for video-text retrieval. HiT performs Hierarchical Cross-modal Contrastive Matching in both feature-level and semantic-level, achieving multi-view and comprehensive retrieval results. Moreover, inspired by MoCo, we propose Momentum Cross-modal Contrast for cross-modal learning to enable large-scale negative sample interactions on-the-fly, which contributes to the generation of more precise and discriminative representations. Experimental results on the three major Video-Text Retrieval benchmark datasets demonstrate the advantages of our method.