Skeletal Human Action Recognition using Hybrid Attention based Graph Convolutional Network

TL;DR

This paper introduces a novel adaptive spatial attention mechanism and a new initial graph adjacency matrix for skeleton-based human action recognition, improving accuracy on challenging datasets.

Contribution

It proposes a hybrid attention-based graph convolutional network with dynamic attention and a redesigned adjacency matrix for better action recognition.

Findings

Enhanced recognition accuracy on NTU-RGB+D and Kinetics datasets.

Effective modeling of dynamic relations between skeletal joints.

Strong performance demonstrated on large-scale datasets.

Abstract

In skeleton-based action recognition, Graph Convolutional Networks model human skeletal joints as vertices and connect them through an adjacency matrix, which can be seen as a local attention mask. However, in most existing Graph Convolutional Networks, the local attention mask is defined based on natural connections of human skeleton joints and ignores the dynamic relations for example between head, hands and feet joints. In addition, the attention mechanism has been proven effective in Natural Language Processing and image description, which is rarely investigated in existing methods. In this work, we proposed a new adaptive spatial attention layer that extends local attention map to global based on relative distance and relative angle information. Moreover, we design a new initial graph adjacency matrix that connects head, hands and feet, which shows visible improvement in terms of action recognition accuracy. The proposed model is evaluated on two large-scale and challenging datasets in the field of human activities in daily life: NTU-RGB+D and Kinetics skeleton. The results demonstrate that our model has strong performance on both dataset.