Fast and Robust Hand Tracking Using Detection-Guided Optimization

TL;DR

This paper introduces a fast, robust hand tracking method using a detection-guided optimization approach with a depth camera, achieving high accuracy and speed suitable for real-time human-computer interaction.

Contribution

The novel detection-guided optimization strategy improves robustness and speed in hand pose estimation, enabling real-time tracking with less computational resources.

Findings

Achieves 50 fps tracking speed without GPU support.

Handles rapid and complex hand articulations.

Works with static or moving cameras.

Abstract

Markerless tracking of hands and fingers is a promising enabler for human-computer interaction. However, adoption has been limited because of tracking inaccuracies, incomplete coverage of motions, low framerate, complex camera setups, and high computational requirements. In this paper, we present a fast method for accurately tracking rapid and complex articulations of the hand using a single depth camera. Our algorithm uses a novel detection-guided optimization strategy that increases the robustness and speed of pose estimation. In the detection step, a randomized decision forest classifies pixels into parts of the hand. In the optimization step, a novel objective function combines the detected part labels and a Gaussian mixture representation of the depth to estimate a pose that best fits the depth. Our approach needs comparably less computational resources which makes it extremely fast (50 fps without GPU support). The approach also supports varying static, or moving, camera-to-scene arrangements. We show the benefits of our method by evaluating on public datasets and comparing against previous work.