Step-by-step Erasion, One-by-one Collection: A Weakly Supervised Temporal Action Detector

TL;DR

This paper introduces a novel weakly supervised temporal action detection method that progressively erases action segments to improve detection accuracy, achieving state-of-the-art results on popular datasets.

Contribution

It proposes a step-by-step erasion training strategy that reconciles classifier and detector roles, enhancing weakly supervised temporal action detection.

Findings

Achieves state-of-the-art results on THUMOS'14 and ActivityNet datasets.

Outperforms many strongly supervised methods.

Demonstrates effective temporal localization refinement.

Abstract

Weakly supervised temporal action detection is a Herculean task in understanding untrimmed videos, since no supervisory signal except the video-level category label is available on training data. Under the supervision of category labels, weakly supervised detectors are usually built upon classifiers. However, there is an inherent contradiction between classifier and detector; i.e., a classifier in pursuit of high classification performance prefers top-level discriminative video clips that are extremely fragmentary, whereas a detector is obliged to discover the whole action instance without missing any relevant snippet. To reconcile this contradiction, we train a detector by driving a series of classifiers to find new actionness clips progressively, via step-by-step erasion from a complete video. During the test phase, all we need to do is to collect detection results from the one-by-one trained classifiers at various erasing steps. To assist in the collection process, a fully connected conditional random field is established to refine the temporal localization outputs. We evaluate our approach on two prevailing datasets, THUMOS'14 and ActivityNet. The experiments show that our detector advances state-of-the-art weakly supervised temporal action detection results, and even compares with quite a few strongly supervised methods.