Three-stream network for enriched Action Recognition

TL;DR

This paper introduces a three-stream CNN architecture with different frame rate pathways and attention mechanisms to improve human action recognition accuracy across multiple datasets.

Contribution

The paper presents a novel three-stream CNN model with multi-rate pathways and attention modules, enhancing action recognition performance over existing methods.

Findings

Achieved state-of-the-art results on UCF-101, Kinetics-600, and AVA datasets.

Demonstrated the effectiveness of multi-rate pathways and attention in capturing spatial and temporal features.

Improved recognition accuracy compared to previous models.

Abstract

Understanding accurate information on human behaviours is one of the most important tasks in machine intelligence. Human Activity Recognition that aims to understand human activities from a video is a challenging task due to various problems including background, camera motion and dataset variations. This paper proposes two CNN based architectures with three streams which allow the model to exploit the dataset under different settings. The three pathways are differentiated in frame rates. The single pathway, operates at a single frame rate captures spatial information, the slow pathway operates at low frame rates captures the spatial information and the fast pathway operates at high frame rates that capture fine temporal information. Post CNN encoders, we add bidirectional LSTM and attention heads respectively to capture the context and temporal features. By experimenting with various algorithms on UCF-101, Kinetics-600 and AVA dataset, we observe that the proposed models achieve state-of-art performance for human action recognition task.