REGINA - Reasoning Graph Convolutional Networks in Human Action Recognition

TL;DR

REGINA enhances human action recognition by integrating handcrafted skeleton features into GCNs, improving performance while maintaining end-to-end trainability and compatibility with existing methods.

Contribution

It introduces a novel approach to incorporate handcrafted features into GCNs for human action recognition, boosting accuracy without altering existing models.

Findings

Significant performance improvements on standard datasets.

Easy integration with existing GCN-based methods.

Maintains end-to-end trainability.

Abstract

It is known that the kinematics of the human body skeleton reveals valuable information in action recognition. Recently, modeling skeletons as spatio-temporal graphs with Graph Convolutional Networks (GCNs) has been reported to solidly advance the state-of-the-art performance. However, GCN-based approaches exclusively learn from raw skeleton data, and are expected to extract the inherent structural information on their own. This paper describes REGINA, introducing a novel way to REasoning Graph convolutional networks IN Human Action recognition. The rationale is to provide to the GCNs additional knowledge about the skeleton data, obtained by handcrafted features, in order to facilitate the learning process, while guaranteeing that it remains fully trainable in an end-to-end manner. The challenge is to capture complementary information over the dynamics between consecutive frames, which is the key information extracted by state-of-the-art GCN techniques. Moreover, the proposed strategy can be easily integrated in the existing GCN-based methods, which we also regard positively. Our experiments were carried out in well known action recognition datasets and enabled to conclude that REGINA contributes for solid improvements in performance when incorporated to other GCN-based approaches, without any other adjustment regarding the original method. For reproducibility, the REGINA code and all the experiments carried out will be publicly available at https://github.com/DegardinBruno.