Combining Spatio-Temporal Appearance Descriptors and Optical Flow for Human Action Recognition in Video Data

TL;DR

This paper explores combining spatio-temporal appearance descriptors with optical flow to improve online human action recognition in videos, validated on the KTH dataset with promising results.

Contribution

It introduces a novel approach integrating dense optical flow with STA descriptors for enhanced human action recognition, including detailed analysis of flow algorithm parameters.

Findings

Effective recognition performance on KTH dataset

Optical flow parameter settings significantly impact results

Potential for real-time online human action recognition

Abstract

This paper proposes combining spatio-temporal appearance (STA) descriptors with optical flow for human action recognition. The STA descriptors are local histogram-based descriptors of space-time, suitable for building a partial representation of arbitrary spatio-temporal phenomena. Because of the possibility of iterative refinement, they are interesting in the context of online human action recognition. We investigate the use of dense optical flow as the image function of the STA descriptor for human action recognition, using two different algorithms for computing the flow: the Farneb\"ack algorithm and the TVL1 algorithm. We provide a detailed analysis of the influencing optical flow algorithm parameters on the produced optical flow fields. An extensive experimental validation of optical flow-based STA descriptors in human action recognition is performed on the KTH human action dataset. The encouraging experimental results suggest the potential of our approach in online human action recognition.