KShapeNet: Riemannian network on Kendall shape space for Skeleton based Action Recognition

TL;DR

This paper introduces a geometry-aware deep learning method for skeleton-based action recognition that models skeleton sequences on Kendall's shape space and uses a CNN-LSTM network, outperforming existing methods.

Contribution

It proposes a novel Riemannian network on Kendall shape space for skeleton data, incorporating geometric transformations into deep learning for improved action recognition.

Findings

Outperforms existing geometric deep learning methods on NTU datasets.

Achieves competitive results with state-of-the-art approaches.

Validates the effectiveness of Riemannian modeling for skeleton data.

Abstract

Deep Learning architectures, albeit successful in most computer vision tasks, were designed for data with an underlying Euclidean structure, which is not usually fulfilled since pre-processed data may lie on a non-linear space. In this paper, we propose a geometry aware deep learning approach for skeleton-based action recognition. Skeleton sequences are first modeled as trajectories on Kendall's shape space and then mapped to the linear tangent space. The resulting structured data are then fed to a deep learning architecture, which includes a layer that optimizes over rigid and non rigid transformations of the 3D skeletons, followed by a CNN-LSTM network. The assessment on two large scale skeleton datasets, namely NTU-RGB+D and NTU-RGB+D 120, has proven that proposed approach outperforms existing geometric deep learning methods and is competitive with respect to recently published approaches.