SHREC 2022 Track on Online Detection of Heterogeneous Gestures

TL;DR

This paper reports on a contest evaluating online heterogeneous gesture recognition from 3D hand pose sequences, highlighting current challenges and the low computational cost of proposed methods in mixed reality contexts.

Contribution

It introduces a benchmark for online heterogeneous gesture detection using Hololens 2 data, emphasizing evaluation metrics like latency and false positives for practical application.

Findings

Recognition errors need reduction for better performance.

Proposed algorithms have sufficiently low computational cost.

Evaluation highlights the importance of latency and false positives.

Abstract

This paper presents the outcomes of a contest organized to evaluate methods for the online recognition of heterogeneous gestures from sequences of 3D hand poses. The task is the detection of gestures belonging to a dictionary of 16 classes characterized by different pose and motion features. The dataset features continuous sequences of hand tracking data where the gestures are interleaved with non-significant motions. The data have been captured using the Hololens 2 finger tracking system in a realistic use-case of mixed reality interaction. The evaluation is based not only on the detection performances but also on the latency and the false positives, making it possible to understand the feasibility of practical interaction tools based on the algorithms proposed. The outcomes of the contest's evaluation demonstrate the necessity of further research to reduce recognition errors, while the computational cost of the algorithms proposed is sufficiently low.