Video Action Recognition Collaborative Learning with Dynamics via PSO-ConvNet Transformer

TL;DR

This paper introduces a novel PSO-ConvNet model combined with Transformers and RNNs for video human action recognition, achieving significant accuracy improvements and demonstrating the benefits of collaborative learning over individual methods.

Contribution

The paper presents a dynamic PSO-ConvNet framework integrated with temporal models for improved video action recognition, extending prior image recognition work to spatio-temporal video analysis.

Findings

Up to 9% accuracy improvement on UCF-101 dataset.

Collaborative learning outperforms non-collaborative approaches.

Effective capture of spatio-temporal dynamics in videos.

Abstract

Recognizing human actions in video sequences, known as Human Action Recognition (HAR), is a challenging task in pattern recognition. While Convolutional Neural Networks (ConvNets) have shown remarkable success in image recognition, they are not always directly applicable to HAR, as temporal features are critical for accurate classification. In this paper, we propose a novel dynamic PSO-ConvNet model for learning actions in videos, building on our recent work in image recognition. Our approach leverages a framework where the weight vector of each neural network represents the position of a particle in phase space, and particles share their current weight vectors and gradient estimates of the Loss function. To extend our approach to video, we integrate ConvNets with state-of-the-art temporal methods such as Transformer and Recurrent Neural Networks. Our experimental results on the UCF-101 dataset demonstrate substantial improvements of up to 9% in accuracy, which confirms the effectiveness of our proposed method. In addition, we conducted experiments on larger and more variety of datasets including Kinetics-400 and HMDB-51 and obtained preference for Collaborative Learning in comparison with Non-Collaborative Learning (Individual Learning). Overall, our dynamic PSO-ConvNet model provides a promising direction for improving HAR by better capturing the spatio-temporal dynamics of human actions in videos. The code is available at https://github.com/leonlha/Video-Action-Recognition-Collaborative-Learning-with-Dynamics-via-PSO-ConvNet-Transformer.