Context-LSTM: a robust classifier for video detection on UCF101

TL;DR

This paper introduces Context-LSTM, a simplified yet effective LSTM-based model for video detection that reduces training time and GPU memory usage while maintaining high accuracy on UCF101.

Contribution

The paper proposes a novel LSTM-based architecture called Context-LSTM that is computationally efficient and achieves competitive accuracy in video detection tasks.

Findings

Reduces training time and GPU memory usage

Achieves top-1 accuracy comparable to state-of-the-art methods

Demonstrates robust performance on UCF101 dataset

Abstract

Video detection and human action recognition may be computationally expensive, and need a long time to train models. In this paper, we were intended to reduce the training time and the GPU memory usage of video detection, and achieved a competitive detection accuracy. Other research works such as Two-stream, C3D, TSN have shown excellent performance on UCF101. Here, we used a LSTM structure simply for video detection. We used a simple structure to perform a competitive top-1 accuracy on the entire validation dataset of UCF101. The LSTM structure is named Context-LSTM, since it may process the deep temporal features. The Context-LSTM may simulate the human recognition system. We cascaded the LSTM blocks in PyTorch and connected the cell state flow and hidden output flow. At the connection of the blocks, we used ReLU, Batch Normalization, and MaxPooling functions. The Context-LSTM could reduce the training time and the GPU memory usage, while keeping a state-of-the-art top-1 accuracy on UCF101 entire validation dataset, show a robust performance on video action detection.