Ensemble One-dimensional Convolution Neural Networks for Skeleton-based Action Recognition

TL;DR

This paper introduces an ensemble of four specialized 1D convolutional neural networks for skeleton-based action recognition, achieving state-of-the-art results by capturing diverse features from skeleton sequences.

Contribution

The paper proposes a novel ensemble framework of four subnets based on a residual 1D CNN, each capturing different aspects of skeleton data, with a focus on diversity and performance.

Findings

Achieved state-of-the-art accuracy on three benchmark datasets.

Demonstrated the effectiveness of multi-aspect feature extraction.

Validated the ensemble approach's superiority over single models.

Abstract

In this paper, we proposed a effective but extensible residual one-dimensional convolution neural network as base network, based on the this network, we proposed four subnets to explore the features of skeleton sequences from each aspect. Given a skeleton sequences, the spatial information are encoded into the skeleton joints coordinate in a frame and the temporal information are present by multiple frames. Limited by the skeleton sequence representations, two-dimensional convolution neural network cannot be used directly, we chose one-dimensional convolution layer as the basic layer. Each sub network could extract discriminative features from different aspects. Our first subnet is a two-stream network which could explore both temporal and spatial information. The second is a body-parted network, which could gain micro spatial features and macro temporal features. The third one is an attention network, the main contribution of which is to focus the key frames and feature channels which high related with the action classes in a skeleton sequence. One frame-difference network, as the last subnet, mainly processes the joints changes between the consecutive frames. Four subnets ensemble together by late fusion, the key problem of ensemble method is each subnet should have a certain performance and between the subnets, there are diversity existing. Each subnet shares a wellperformance basenet and differences between subnets guaranteed the diversity. Experimental results show that the ensemble network gets a state-of-the-art performance on three widely used datasets.