Make Skeleton-based Action Recognition Model Smaller, Faster and Better

TL;DR

This paper introduces DD-Net, a lightweight and fast skeleton-based action recognition model that achieves state-of-the-art accuracy while maintaining high processing speeds on GPUs and CPUs.

Contribution

The paper proposes a novel lightweight network architecture, DD-Net, that significantly improves speed and maintains high accuracy in skeleton-based action recognition.

Findings

Achieves 3,500 FPS on GPU and 2,000 FPS on CPU.

Uses only 0.15 million parameters.

Outperforms existing methods on SHREC and JHMDB datasets.

Abstract

Although skeleton-based action recognition has achieved great success in recent years, most of the existing methods may suffer from a large model size and slow execution speed. To alleviate this issue, we analyze skeleton sequence properties to propose a Double-feature Double-motion Network (DD-Net) for skeleton-based action recognition. By using a lightweight network structure (i.e., 0.15 million parameters), DD-Net can reach a super fast speed, as 3,500 FPS on one GPU, or, 2,000 FPS on one CPU. By employing robust features, DD-Net achieves the state-of-the-art performance on our experimental datasets: SHREC (i.e., hand actions) and JHMDB (i.e., body actions). Our code will be released with this paper later.