PcmNet: Position-Sensitive Context Modeling Network for Temporal Action Localization

TL;DR

PcmNet introduces a position-sensitive context modeling approach that combines positional and semantic cues to improve the accuracy of temporal action localization in videos, achieving state-of-the-art results.

Contribution

The paper proposes a novel temporal-position-sensitive context modeling method that incorporates directed temporal positional encoding and attention mechanisms for better action boundary detection.

Findings

Achieves state-of-the-art performance on THUMOS-14 and ActivityNet-1.3 datasets.

Effectively encodes position-aware context for improved localization.

Enhances boundary detection and proposal evaluation accuracy.

Abstract

Temporal action localization is an important and challenging task that aims to locate temporal regions in real-world untrimmed videos where actions occur and recognize their classes. It is widely acknowledged that video context is a critical cue for video understanding, and exploiting the context has become an important strategy to boost localization performance. However, previous state-of-the-art methods focus more on exploring semantic context which captures the feature similarity among frames or proposals, and neglect positional context which is vital for temporal localization. In this paper, we propose a temporal-position-sensitive context modeling approach to incorporate both positional and semantic information for more precise action localization. Specifically, we first augment feature representations with directed temporal positional encoding, and then conduct attention-based information propagation, in both frame-level and proposal-level. Consequently, the generated feature representations are significantly empowered with the discriminative capability of encoding the position-aware context information, and thus benefit boundary detection and proposal evaluation. We achieve state-of-the-art performance on both two challenging datasets, THUMOS-14 and ActivityNet-1.3, demonstrating the effectiveness and generalization ability of our method.