Weakly-Supervised Completion Moment Detection using Temporal Attention

TL;DR

This paper introduces a weakly-supervised method for detecting the completion moment of actions in videos using temporal attention, reducing the need for detailed annotations and improving detection accuracy across multiple datasets.

Contribution

It presents a novel approach that leverages weak video-level labels to learn temporal attention for completion detection, applicable in both weakly- and fully-supervised scenarios.

Findings

Temporal attention improves detection accuracy.

Method performs well on multiple datasets.

Effective with limited supervision.

Abstract

Monitoring the progression of an action towards completion offers fine grained insight into the actor's behaviour. In this work, we target detecting the completion moment of actions, that is the moment when the action's goal has been successfully accomplished. This has potential applications from surveillance to assistive living and human-robot interactions. Previous effort required human annotations of the completion moment for training (i.e. full supervision). In this work, we present an approach for moment detection from weak video-level labels. Given both complete and incomplete sequences, of the same action, we learn temporal attention, along with accumulated completion prediction from all frames in the sequence. We also demonstrate how the approach can be used when completion moment supervision is available. We evaluate and compare our approach on actions from three datasets, namely HMDB, UCF101 and RGBD-AC, and show that temporal attention improves detection in both weakly-supervised and fully-supervised settings.