Weakly Supervised Temporal Action Localization Through Learning Explicit Subspaces for Action and Context

TL;DR

This paper introduces a novel weakly supervised temporal action localization framework that explicitly learns separate feature subspaces for actions and context, improving localization accuracy by reducing action-context confusion.

Contribution

It proposes a method to learn distinct subspaces for action and context features using only video-level labels, enhancing localization precision in WS-TAL.

Findings

Outperforms state-of-the-art methods on THUMOS14, ActivityNet v1.2 and v1.3 datasets.

Effectively reduces action-context confusion in localization.

Utilizes an unsupervised learning task to focus on temporal information.

Abstract

Weakly-supervised Temporal Action Localization (WS-TAL) methods learn to localize temporal starts and ends of action instances in a video under only video-level supervision. Existing WS-TAL methods rely on deep features learned for action recognition. However, due to the mismatch between classification and localization, these features cannot distinguish the frequently co-occurring contextual background, i.e., the context, and the actual action instances. We term this challenge action-context confusion, and it will adversely affect the action localization accuracy. To address this challenge, we introduce a framework that learns two feature subspaces respectively for actions and their context. By explicitly accounting for action visual elements, the action instances can be localized more precisely without the distraction from the context. To facilitate the learning of these two feature subspaces with only video-level categorical labels, we leverage the predictions from both spatial and temporal streams for snippets grouping. In addition, an unsupervised learning task is introduced to make the proposed module focus on mining temporal information. The proposed approach outperforms state-of-the-art WS-TAL methods on three benchmarks, i.e., THUMOS14, ActivityNet v1.2 and v1.3 datasets.