Part-based Graph Convolutional Network for Action Recognition

TL;DR

This paper proposes a part-based graph convolutional network that divides the skeleton into subgraphs, improving action recognition accuracy on benchmark datasets by using relative coordinates and temporal displacements.

Contribution

The paper introduces a novel part-based GCN for skeletal action recognition, inspired by DPMs, with shared joints across subgraphs and enhanced node features.

Findings

Achieves state-of-the-art results on NTURGB+D and HDM05 datasets.

Using relative coordinates and temporal displacements improves recognition performance.

Part-based modeling outperforms whole-skeleton approaches.

Abstract

Human actions comprise of joint motion of articulated body parts or `gestures'. Human skeleton is intuitively represented as a sparse graph with joints as nodes and natural connections between them as edges. Graph convolutional networks have been used to recognize actions from skeletal videos. We introduce a part-based graph convolutional network (PB-GCN) for this task, inspired by Deformable Part-based Models (DPMs). We divide the skeleton graph into four subgraphs with joints shared across them and learn a recognition model using a part-based graph convolutional network. We show that such a model improves performance of recognition, compared to a model using entire skeleton graph. Instead of using 3D joint coordinates as node features, we show that using relative coordinates and temporal displacements boosts performance. Our model achieves state-of-the-art performance on two challenging benchmark datasets NTURGB+D and HDM05, for skeletal action recognition.