ACGNet: Action Complement Graph Network for Weakly-supervised Temporal Action Localization

TL;DR

This paper introduces ACGNet, a graph convolutional network that enhances segment features for weakly-supervised temporal action localization, leading to improved accuracy by capturing spatial-temporal dependencies.

Contribution

The paper proposes a novel ACGNet module that can be integrated into existing WTAL frameworks to improve segment feature discrimination and robustness.

Findings

Achieves state-of-the-art results on THUMOS'14 and ActivityNet1.2.

Demonstrates the effectiveness of the graph-based approach in capturing dependencies.

Shows compatibility of ACGNet with various WTAL frameworks.

Abstract

Weakly-supervised temporal action localization (WTAL) in untrimmed videos has emerged as a practical but challenging task since only video-level labels are available. Existing approaches typically leverage off-the-shelf segment-level features, which suffer from spatial incompleteness and temporal incoherence, thus limiting their performance. In this paper, we tackle this problem from a new perspective by enhancing segment-level representations with a simple yet effective graph convolutional network, namely action complement graph network (ACGNet). It facilitates the current video segment to perceive spatial-temporal dependencies from others that potentially convey complementary clues, implicitly mitigating the negative effects caused by the two issues above. By this means, the segment-level features are more discriminative and robust to spatial-temporal variations, contributing to higher localization accuracies. More importantly, the proposed ACGNet works as a universal module that can be flexibly plugged into different WTAL frameworks, while maintaining the end-to-end training fashion. Extensive experiments are conducted on the THUMOS'14 and ActivityNet1.2 benchmarks, where the state-of-the-art results clearly demonstrate the superiority of the proposed approach.