Precise Temporal Action Localization by Evolving Temporal Proposals

TL;DR

This paper introduces a three-phase framework for precise temporal action localization in videos, utilizing multi-stage refinement and a novel non-local pyramid feature to improve boundary accuracy.

Contribution

It presents a new multi-stage refinement framework with an Actionness Network, Refinement Network, and Localization Network, incorporating a non-local pyramid feature for better temporal localization.

Findings

Achieves 34.2% mAP@IoU=0.5 on THUMOS14

Significant improvement over state-of-the-art methods at high IoU thresholds

Effective boundary refinement through multi-stage processing

Abstract

Locating actions in long untrimmed videos has been a challenging problem in video content analysis. The performances of existing action localization approaches remain unsatisfactory in precisely determining the beginning and the end of an action. Imitating the human perception procedure with observations and refinements, we propose a novel three-phase action localization framework. Our framework is embedded with an Actionness Network to generate initial proposals through frame-wise similarity grouping, and then a Refinement Network to conduct boundary adjustment on these proposals. Finally, the refined proposals are sent to a Localization Network for further fine-grained location regression. The whole process can be deemed as multi-stage refinement using a novel non-local pyramid feature under various temporal granularities. We evaluate our framework on THUMOS14 benchmark and obtain a significant improvement over the state-of-the-arts approaches. Specifically, the performance gain is remarkable under precise localization with high IoU thresholds. Our proposed framework achieves mAP@IoU=0.5 of 34.2%.