SMART-Vision: Survey of Modern Action Recognition Techniques in Vision

TL;DR

This survey introduces the SMART-Vision taxonomy for human action recognition, emphasizing hybrid deep learning approaches, datasets, and emerging open-set challenges in the field.

Contribution

It presents a novel taxonomy that categorizes hybrid HAR methods and discusses how different architectures and modalities are integrated in modern systems.

Findings

Highlights the importance of hybrid approaches in HAR

Provides a comprehensive overview of datasets used in HAR

Discusses emerging open-HAR challenges and future directions

Abstract

Human Action Recognition (HAR) is a challenging domain in computer vision, involving recognizing complex patterns by analyzing the spatiotemporal dynamics of individuals' movements in videos. These patterns arise in sequential data, such as video frames, which are often essential to accurately distinguish actions that would be ambiguous in a single image. HAR has garnered considerable interest due to its broad applicability, ranging from robotics and surveillance systems to sports motion analysis, healthcare, and the burgeoning field of autonomous vehicles. While several taxonomies have been proposed to categorize HAR approaches in surveys, they often overlook hybrid methodologies and fail to demonstrate how different models incorporate various architectures and modalities. In this comprehensive survey, we present the novel SMART-Vision taxonomy, which illustrates how innovations in deep learning for HAR complement one another, leading to hybrid approaches beyond traditional categories. Our survey provides a clear roadmap from foundational HAR works to current state-of-the-art systems, highlighting emerging research directions and addressing unresolved challenges in discussion sections for architectures within the HAR domain. We provide details of the research datasets that various approaches used to measure and compare goodness HAR approaches. We also explore the rapidly emerging field of Open-HAR systems, which challenges HAR systems by presenting samples from unknown, novel classes during test time.