Spatio-Temporal Pyramid Graph Convolutions for Human Action Recognition and Postural Assessment

TL;DR

This paper introduces a novel spatio-temporal graph convolutional network for online human action recognition, improving ergonomic risk assessment by integrating skeleton features and outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a new Spatio-Temporal Pyramid Graph Convolutional Network that leverages multi-level skeleton features for enhanced online action recognition in ergonomic assessment.

Findings

Outperforms state-of-the-art algorithms on TUM and UW-IOM datasets

Enables improved postural assessment with online action recognition

Integrates with ergonomic risk index (REBA) for musculoskeletal disorder evaluation

Abstract

Recognition of human actions and associated interactions with objects and the environment is an important problem in computer vision due to its potential applications in a variety of domains. The most versatile methods can generalize to various environments and deal with cluttered backgrounds, occlusions, and viewpoint variations. Among them, methods based on graph convolutional networks that extract features from the skeleton have demonstrated promising performance. In this paper, we propose a novel Spatio-Temporal Pyramid Graph Convolutional Network (ST-PGN) for online action recognition for ergonomic risk assessment that enables the use of features from all levels of the skeleton feature hierarchy. The proposed algorithm outperforms state-of-art action recognition algorithms tested on two public benchmark datasets typically used for postural assessment (TUM and UW-IOM). We also introduce a pipeline to enhance postural assessment methods with online action recognition techniques. Finally, the proposed algorithm is integrated with a traditional ergonomic risk index (REBA) to demonstrate the potential value for assessment of musculoskeletal disorders in occupational safety.