A Fine-to-Coarse Convolutional Neural Network for 3D Human Action Recognition

TL;DR

This paper introduces a fine-to-coarse CNN framework that effectively captures temporal and spatial features from 3D skeleton sequences, improving human action recognition accuracy, especially in two-person interactions.

Contribution

The study proposes a novel F2C CNN architecture that segments skeleton sequences to better exploit temporal correlations, achieving competitive results on benchmark datasets.

Findings

Achieves 79.6% accuracy on NTU RGB+D cross-object protocol.

Achieves 84.6% accuracy on NTU RGB+D cross-view protocol.

Significantly improves recognition of two-person interactions.

Abstract

This paper presents a new framework for human action recognition from a 3D skeleton sequence. Previous studies do not fully utilize the temporal relationships between video segments in a human action. Some studies successfully used very deep Convolutional Neural Network (CNN) models but often suffer from the data insufficiency problem. In this study, we first segment a skeleton sequence into distinct temporal segments in order to exploit the correlations between them. The temporal and spatial features of a skeleton sequence are then extracted simultaneously by utilizing a fine-to-coarse (F2C) CNN architecture optimized for human skeleton sequences. We evaluate our proposed method on NTU RGB+D and SBU Kinect Interaction dataset. It achieves 79.6% and 84.6% of accuracies on NTU RGB+D with cross-object and cross-view protocol, respectively, which are almost identical with the state-of-the-art performance. In addition, our method significantly improves the accuracy of the actions in two-person interactions.