Learning Temporal Action Proposals With Fewer Labels

TL;DR

This paper introduces a semi-supervised learning method for temporal action proposal networks that requires fewer annotations and outperforms fully supervised methods on challenging video datasets.

Contribution

The work presents a novel semi-supervised training algorithm for temporal action proposals that reduces annotation costs while maintaining or improving performance.

Findings

Semi-supervised method outperforms fully supervised approaches with limited labels.

Consistently matches or exceeds state-of-the-art on ActivityNet v1.3 and THUMOS14.

Reduces annotation effort in training temporal action proposal models.

Abstract

Temporal action proposals are a common module in action detection pipelines today. Most current methods for training action proposal modules rely on fully supervised approaches that require large amounts of annotated temporal action intervals in long video sequences. The large cost and effort in annotation that this entails motivate us to study the problem of training proposal modules with less supervision. In this work, we propose a semi-supervised learning algorithm specifically designed for training temporal action proposal networks. When only a small number of labels are available, our semi-supervised method generates significantly better proposals than the fully-supervised counterpart and other strong semi-supervised baselines. We validate our method on two challenging action detection video datasets, ActivityNet v1.3 and THUMOS14. We show that our semi-supervised approach consistently matches or outperforms the fully supervised state-of-the-art approaches.