Weakly-Supervised Action Localization by Hierarchically-structured Latent Attention Modeling

TL;DR

This paper introduces a hierarchical latent attention model for weakly-supervised action localization in videos, effectively capturing temporal feature variations to improve localization accuracy.

Contribution

It proposes a novel hierarchical model combining change-point detection and attention-based classification to better handle temporal variations in weakly-supervised settings.

Findings

Outperforms current state-of-the-art methods on THUMOS-14 and ActivityNet-v1.3 datasets.

Achieves performance comparable to fully-supervised methods.

Effectively models temporal feature variations for improved localization.

Abstract

Weakly-supervised action localization aims to recognize and localize action instancese in untrimmed videos with only video-level labels. Most existing models rely on multiple instance learning(MIL), where the predictions of unlabeled instances are supervised by classifying labeled bags. The MIL-based methods are relatively well studied with cogent performance achieved on classification but not on localization. Generally, they locate temporal regions by the video-level classification but overlook the temporal variations of feature semantics. To address this problem, we propose a novel attention-based hierarchically-structured latent model to learn the temporal variations of feature semantics. Specifically, our model entails two components, the first is an unsupervised change-points detection module that detects change-points by learning the latent representations of video features in a temporal hierarchy based on their rates of change, and the second is an attention-based classification model that selects the change-points of the foreground as the boundaries. To evaluate the effectiveness of our model, we conduct extensive experiments on two benchmark datasets, THUMOS-14 and ActivityNet-v1.3. The experiments show that our method outperforms current state-of-the-art methods, and even achieves comparable performance with fully-supervised methods.