Real-time Action Recognition for Fine-Grained Actions and The Hand Wash Dataset

TL;DR

This paper introduces a real-time three-stream action recognition algorithm that effectively distinguishes fine-grained actions and presents a new handwash video dataset to advance research in this area.

Contribution

The paper proposes a novel, efficient three-stream fusion algorithm capable of real-time recognition of subtle, similar actions and introduces the Hand Wash Dataset for fine-grained action benchmarking.

Findings

Achieved 92.7% accuracy on UCF-101 dataset.

Achieved 64.9% accuracy on HMDB-51 dataset.

Demonstrated real-time performance on low-powered devices.

Abstract

In this paper we present a three-stream algorithm for real-time action recognition and a new dataset of handwash videos, with the intent of aligning action recognition with real-world constraints to yield effective conclusions. A three-stream fusion algorithm is proposed, which runs both accurately and efficiently, in real-time even on low-powered systems such as a Raspberry Pi. The cornerstone of the proposed algorithm is the incorporation of both spatial and temporal information, as well as the information of the objects in a video while using an efficient architecture, and Optical Flow computation to achieve commendable results in real-time. The results achieved by this algorithm are benchmarked on the UCF-101 as well as the HMDB-51 datasets, achieving an accuracy of 92.7% and 64.9% respectively. An important point to note is that the algorithm is novel in the aspect that it is also able to learn the intricate differences between extremely similar actions, which would be difficult even for the human eye. Additionally, noticing a dearth in the number of datasets for the recognition of very similar or fine-grained actions, this paper also introduces a new dataset that is made publicly available, the Hand Wash Dataset with the intent of introducing a new benchmark for fine-grained action recognition tasks in the future.