Skeleton Split Strategies for Spatial Temporal Graph Convolution Networks

TL;DR

This paper proposes new skeleton split strategies for spatial-temporal graph convolutional networks, improving performance and stability in human activity recognition tasks without extra training parameters.

Contribution

It introduces three novel skeleton split processes that outperform previous methods in ST-GCN frameworks for action recognition.

Findings

All split processes outperform previous strategies.

The methods are more stable during training.

No additional edge importance weighting needed.

Abstract

A skeleton representation of the human body has been proven to be effective for this task. The skeletons are presented in graphs form-like. However, the topology of a graph is not structured like Euclidean-based data. Therefore, a new set of methods to perform the convolution operation upon the skeleton graph is presented. Our proposal is based upon the ST-GCN framework proposed by Yan et al. [1]. In this study, we present an improved set of label mapping methods for the ST-GCN framework. We introduce three split processes (full distance split, connection split, and index split) as an alternative approach for the convolution operation. To evaluate the performance, the experiments presented in this study have been trained using two benchmark datasets: NTU-RGB+D and Kinetics. Our results indicate that all of our split processes outperform the previous partition strategies and are more stable during training without using the edge importance weighting additional training parameter. Therefore, our proposal can provide a more realistic solution for real-time applications centred on daily living recognition systems activities for indoor environments.