Deep-Aligned Convolutional Neural Network for Skeleton-based Action Recognition and Segmentation

TL;DR

This paper introduces a novel deep-aligned CNN (DACNN) for skeleton-based action recognition and segmentation, effectively addressing spatial relationship and temporal scaling challenges with interpretable filters, achieving competitive results.

Contribution

The paper proposes a new type of alignment-based filters in CNNs that improve interpretability and performance in SBARS tasks, handling non-uniform temporal scales.

Findings

DACNN achieves competitive performance on real-world benchmarks.

The model offers improved interpretability over traditional CNNs.

It effectively manages non-uniform temporal scalings in skeleton data.

Abstract

Convolutional neural networks (CNNs) are deep learning frameworks which are well-known for their notable performance in classification tasks. Hence, many skeleton-based action recognition and segmentation (SBARS) algorithms benefit from them in their designs. However, a shortcoming of such applications is the general lack of spatial relationships between the input features in such data types. Besides, non-uniform temporal scalings is a common issue in skeleton-based data streams which leads to having different input sizes even within one specific action category. In this work, we propose a novel deep-aligned convolutional neural network (DACNN) to tackle the above challenges for the particular problem of SBARS. Our network is designed by introducing a new type of filters in the context of CNNs which are trained based on their alignments to the local subsequences in the inputs. These filters result in efficient predictions as well as learning interpretable patterns in the data. We empirically evaluate our framework on real-world benchmarks showing that the proposed DACNN algorithm obtains a competitive performance compared to the state-of-the-art while benefiting from a less complicated yet more interpretable model.