Efficient Hand Articulations Tracking using Adaptive Hand Model and Depth map

TL;DR

This paper presents a real-time, GPU-free hand tracking system that adapts to different hand shapes and uses depth maps for accurate articulation tracking, suitable for mobile and wearable devices.

Contribution

It introduces an adaptive hand model, improved initialization, and hierarchical pixel sampling to enable efficient, personalized hand tracking without high-performance GPUs.

Findings

Achieves real-time hand tracking without GPUs

Handles different hand shapes without personalization

Improves accuracy with hierarchical pixel sampling

Abstract

Real-time hand articulations tracking is important for many applications such as interacting with virtual / augmented reality devices or tablets. However, most of existing algorithms highly rely on expensive and high power-consuming GPUs to achieve real-time processing. Consequently, these systems are inappropriate for mobile and wearable devices. In this paper, we propose an efficient hand tracking system which does not require high performance GPUs. In our system, we track hand articulations by minimizing discrepancy between depth map from sensor and computer-generated hand model. We also initialize hand pose at each frame using finger detection and classification. Our contributions are: (a) propose adaptive hand model to consider different hand shapes of users without generating personalized hand model; (b) improve the highly efficient frame initialization for robust tracking and automatic initialization; (c) propose hierarchical random sampling of pixels from each depth map to improve tracking accuracy while limiting required computations. To the best of our knowledge, it is the first system that achieves both automatic hand model adjustment and real-time tracking without using GPUs.