Real-Time Hand Tracking Using a Sum of Anisotropic Gaussians Model

TL;DR

This paper introduces a real-time, marker-less hand tracking method using a novel Sum of Anisotropic Gaussians shape model, achieving higher accuracy and 25 fps performance from multiple RGB cameras.

Contribution

It presents a new generative hand tracking approach with an implicit shape model and differentiable pose fitting energy, improving accuracy over previous methods.

Findings

Achieves real-time tracking at 25 fps.

Outperforms previous methods in accuracy.

Validated on publicly available datasets.

Abstract

Real-time marker-less hand tracking is of increasing importance in human-computer interaction. Robust and accurate tracking of arbitrary hand motion is a challenging problem due to the many degrees of freedom, frequent self-occlusions, fast motions, and uniform skin color. In this paper, we propose a new approach that tracks the full skeleton motion of the hand from multiple RGB cameras in real-time. The main contributions include a new generative tracking method which employs an implicit hand shape representation based on Sum of Anisotropic Gaussians (SAG), and a pose fitting energy that is smooth and analytically differentiable making fast gradient based pose optimization possible. This shape representation, together with a full perspective projection model, enables more accurate hand modeling than a related baseline method from literature. Our method achieves better accuracy than previous methods and runs at 25 fps. We show these improvements both qualitatively and quantitatively on publicly available datasets.