No-frills Temporal Video Grounding: Multi-Scale Neighboring Attention and Zoom-in Boundary Detection

TL;DR

This paper introduces a lightweight, no-frills temporal video grounding model that uses multi-scale neighboring attention and zoom-in boundary detection to improve accuracy and speed in localizing language queries in videos.

Contribution

The paper presents a novel, simple TVG model with multi-scale neighboring attention and zoom-in boundary detection, achieving competitive results with faster inference and fewer parameters.

Findings

Achieves competitive performance on TVG benchmarks.

Faster inference speed due to lightweight architecture.

Effective in low SNR scenarios with minimal noise.

Abstract

Temporal video grounding (TVG) aims to retrieve the time interval of a language query from an untrimmed video. A significant challenge in TVG is the low "Semantic Noise Ratio (SNR)", which results in worse performance with lower SNR. Prior works have addressed this challenge using sophisticated techniques. In this paper, we propose a no-frills TVG model that consists of two core modules, namely multi-scale neighboring attention and zoom-in boundary detection. The multi-scale neighboring attention restricts each video token to only aggregate visual contexts from its neighbor, enabling the extraction of the most distinguishing information with multi-scale feature hierarchies from high-ratio noises. The zoom-in boundary detection then focuses on local-wise discrimination of the selected top candidates for fine-grained grounding adjustment. With an end-to-end training strategy, our model achieves competitive performance on different TVG benchmarks, while also having the advantage of faster inference speed and lighter model parameters, thanks to its lightweight architecture.