Local Spherical Harmonics Improve Skeleton-Based Hand Action Recognition

TL;DR

This paper introduces a novel hand action recognition method using local Spherical Harmonics and angular embeddings, enhancing robustness to viewpoint and subject variations in skeleton-based recognition tasks.

Contribution

It proposes a new hand representation technique with rotation-invariant properties using local Spherical Harmonics, improving recognition accuracy over existing methods.

Findings

Enhanced recognition robustness against viewpoint changes

Improved accuracy on benchmark datasets

Effective hand action representation with local Spherical Harmonics

Abstract

Hand action recognition is essential. Communication, human-robot interactions, and gesture control are dependent on it. Skeleton-based action recognition traditionally includes hands, which belong to the classes which remain challenging to correctly recognize to date. We propose a method specifically designed for hand action recognition which uses relative angular embeddings and local Spherical Harmonics to create novel hand representations. The use of Spherical Harmonics creates rotation-invariant representations which make hand action recognition even more robust against inter-subject differences and viewpoint changes. We conduct extensive experiments on the hand joints in the First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations, and on the NTU RGB+D 120 dataset, demonstrating the benefit of using Local Spherical Harmonics Representations. Our code is available at https://github.com/KathPra/LSHR_LSHT.