SAR-NAS: Skeleton-based Action Recognition via Neural Architecture Searching

TL;DR

This paper introduces SAR-NAS, an automated neural architecture search method for skeleton-based action recognition, resulting in compact, efficient networks that outperform many existing methods on benchmark datasets.

Contribution

It applies differentiable architecture search to design efficient skeleton-based action recognition networks, reducing computation and improving performance.

Findings

Achieved comparable or better accuracy than state-of-the-art methods.

Produced more compact and efficient network architectures.

Validated on NTU RGB+D and Kinectics datasets.

Abstract

This paper presents a study of automatic design of neural network architectures for skeleton-based action recognition. Specifically, we encode a skeleton-based action instance into a tensor and carefully define a set of operations to build two types of network cells: normal cells and reduction cells. The recently developed DARTS (Differentiable Architecture Search) is adopted to search for an effective network architecture that is built upon the two types of cells. All operations are 2D based in order to reduce the overall computation and search space. Experiments on the challenging NTU RGB+D and Kinectics datasets have verified that most of the networks developed to date for skeleton-based action recognition are likely not compact and efficient. The proposed method provides an approach to search for such a compact network that is able to achieve comparative or even better performance than the state-of-the-art methods.