An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition

TL;DR

This paper introduces an Attention Enhanced Graph Convolutional LSTM Network that effectively captures spatial and temporal features for skeleton-based action recognition, outperforming existing methods on benchmark datasets.

Contribution

The paper proposes a novel AGC-LSTM model with attention mechanisms and hierarchical architecture to improve discriminative feature extraction and reduce computation in action recognition.

Findings

Outperforms state-of-the-art methods on NTU RGB+D dataset

Effective in capturing spatial-temporal features and joint importance

Reduces computational cost through hierarchical design

Abstract

Skeleton-based action recognition is an important task that requires the adequate understanding of movement characteristics of a human action from the given skeleton sequence. Recent studies have shown that exploring spatial and temporal features of the skeleton sequence is vital for this task. Nevertheless, how to effectively extract discriminative spatial and temporal features is still a challenging problem. In this paper, we propose a novel Attention Enhanced Graph Convolutional LSTM Network (AGC-LSTM) for human action recognition from skeleton data. The proposed AGC-LSTM can not only capture discriminative features in spatial configuration and temporal dynamics but also explore the co-occurrence relationship between spatial and temporal domains. We also present a temporal hierarchical architecture to increases temporal receptive fields of the top AGC-LSTM layer, which boosts the ability to learn the high-level semantic representation and significantly reduces the computation cost. Furthermore, to select discriminative spatial information, the attention mechanism is employed to enhance information of key joints in each AGC-LSTM layer. Experimental results on two datasets are provided: NTU RGB+D dataset and Northwestern-UCLA dataset. The comparison results demonstrate the effectiveness of our approach and show that our approach outperforms the state-of-the-art methods on both datasets.