Multi Scale Temporal Graph Networks For Skeleton-based Action Recognition

TL;DR

This paper introduces a Multi-Scale Temporal Graph Network (TGN) for skeleton-based action recognition, capturing spatiotemporal features more effectively by integrating multi-scale graphs and a unified representation.

Contribution

The paper proposes a novel TGN model with a multi-scale graph strategy and a generic skeleton sequence representation, addressing limitations of existing GCN-based methods.

Findings

TGN outperforms state-of-the-art methods on large datasets.

Multi-scale graph strategy improves joint relation modeling.

Unified spatiotemporal feature extraction enhances recognition accuracy.

Abstract

Graph convolutional networks (GCNs) can effectively capture the features of related nodes and improve the performance of the model. More attention is paid to employing GCN in Skeleton-Based action recognition. But existing methods based on GCNs have two problems. First, the consistency of temporal and spatial features is ignored for extracting features node by node and frame by frame. To obtain spatiotemporal features simultaneously, we design a generic representation of skeleton sequences for action recognition and propose a novel model called Temporal Graph Networks (TGN). Secondly, the adjacency matrix of the graph describing the relation of joints is mostly dependent on the physical connection between joints. To appropriately describe the relations between joints in the skeleton graph, we propose a multi-scale graph strategy, adopting a full-scale graph, part-scale graph, and core-scale graph to capture the local features of each joint and the contour features of important joints. Experiments were carried out on two large datasets and results show that TGN with our graph strategy outperforms state-of-the-art methods.