Temporal Action Localization Using Gated Recurrent Units

TL;DR

This paper introduces a novel Gated Recurrent Unit-based network with innovative post-processing techniques for temporal action localization, achieving superior accuracy on benchmark datasets compared to existing methods.

Contribution

The paper presents a new GRU-based network architecture with a specialized output layer, linear interpolation for proposal generation, and a Learn to Rank approach for proposal ranking in TAL.

Findings

Achieved 27.52% mAP at IoU 0.7 on Thumos14, outperforming state-of-the-art by 5.12%.

Demonstrated improved accuracy on ActivityNet-1.3 dataset.

Validated effectiveness of the proposed methods through extensive experiments.

Abstract

Temporal Action Localization (TAL) task which is to predict the start and end of each action in a video along with the class label of the action has numerous applications in the real world. But due to the complexity of this task, acceptable accuracy rates have not been achieved yet, whereas this is not the case regarding the action recognition task. In this paper, we propose a new network based on Gated Recurrent Unit (GRU) and two novel post-processing methods for TAL task. Specifically, we propose a new design for the output layer of the conventionally GRU resulting in the so-called GRU-Split network. Moreover, linear interpolation is used to generate the action proposals with precise start and end times. Finally, to rank the generated proposals appropriately, we use a Learn to Rank (LTR) approach. We evaluated the performance of the proposed method on Thumos14 and ActivityNet-1.3 datasets. Results show the superiority of the performance of the proposed method compared to state-of-the-art. Specifically in the mean Average Precision (mAP) metric at Intersection over Union (IoU) of 0.7 on Thumos14, we get 27.52% accuracy which is 5.12% better than that of state-of-the-art methods.