Towards Diverse Temporal Grounding under Single Positive Labels

TL;DR

This paper introduces a novel framework for temporal grounding in videos that accounts for multiple moments described by a single query, using positive moment estimation and diverse regression to improve retrieval accuracy.

Contribution

It reformulates temporal grounding as a one-vs-many problem, proposing the DTG-SPL framework with modules for positive moment estimation and diverse moment regression.

Findings

Outperforms existing methods on Charades-STA and ActivityNet Captions datasets.

Effectively mines potential positive moments to handle multiple relevant video segments.

Achieves superior results in both single-label and multi-label evaluation metrics.

Abstract

Temporal grounding aims to retrieve moments of the described event within an untrimmed video by a language query. Typically, existing methods assume annotations are precise and unique, yet one query may describe multiple moments in many cases. Hence, simply taking it as a one-vs-one mapping task and striving to match single-label annotations will inevitably introduce false negatives during optimization. In this study, we reformulate this task as a one-vs-many optimization problem under the condition of single positive labels. The unlabeled moments are considered unobserved rather than negative, and we explore mining potential positive moments to assist in multiple moment retrieval. In this setting, we propose a novel Diverse Temporal Grounding framework, termed DTG-SPL, which mainly consists of a positive moment estimation (PME) module and a diverse moment regression (DMR) module. PME leverages semantic reconstruction information and an expected positive regularization to uncover potential positive moments in an online fashion. Under the supervision of these pseudo positives, DMR is able to localize diverse moments in parallel that meet different users. The entire framework allows for end-to-end optimization as well as fast inference. Extensive experiments on Charades-STA and ActivityNet Captions show that our method achieves superior performance in terms of both single-label and multi-label metrics.