A New Adjacency Matrix Configuration in GCN-based Models for Skeleton-based Action Recognition

TL;DR

This paper proposes a novel adjacency matrix for GCN models in skeleton-based action recognition, replacing the traditional human skeleton structure with an adaptive approach that improves performance and robustness.

Contribution

It introduces a new adjacency matrix that abandons rigid connections, allowing the model to learn joint relationships adaptively, enhancing recognition accuracy.

Findings

The new adjacency matrix outperforms traditional structures in accuracy.

It improves noise robustness in action recognition.

The approach shows better transferability across datasets.

Abstract

Human skeleton data has received increasing attention in action recognition due to its background robustness and high efficiency. In skeleton-based action recognition, graph convolutional network (GCN) has become the mainstream method. This paper analyzes the fundamental factor for GCN-based models -- the adjacency matrix. We notice that most GCN-based methods conduct their adjacency matrix based on the human natural skeleton structure. Based on our former work and analysis, we propose that the human natural skeleton structure adjacency matrix is not proper for skeleton-based action recognition. We propose a new adjacency matrix that abandons all rigid neighbor connections but lets the model adaptively learn the relationships of joints. We conduct extensive experiments and analysis with a validation model on two skeleton-based action recognition datasets (NTURGBD60 and FineGYM). Comprehensive experimental results and analysis reveals that 1) the most widely used human natural skeleton structure adjacency matrix is unsuitable in skeleton-based action recognition; 2) The proposed adjacency matrix is superior in model performance, noise robustness and transferability.