Hand Segmentation and Fingertip Tracking from Depth Camera Images Using Deep Convolutional Neural Network and Multi-task SegNet

TL;DR

This paper introduces a real-time capable deep learning system for hand segmentation and fingertip detection from RGB-D images, enhancing human-machine interaction applications.

Contribution

It presents a multi-task deep neural network based on a modified SegNet architecture that simultaneously performs hand segmentation and fingertip detection.

Findings

Achieves comparable performance to existing methods.

Suitable for real-time applications due to lightweight model.

Effective in discriminating hand components and locating fingertips.

Abstract

Hand segmentation and fingertip detection play an indispensable role in hand gesture-based human-machine interaction systems. In this study, we propose a method to discriminate hand components and to locate fingertips in RGB-D images. The system consists of three main steps: hand detection using RGB images providing regions which are considered as promising areas for further processing, hand segmentation, and fingertip detection using depth image and our modified SegNet, a single lightweight architecture that can process two independent tasks at the same time. The experimental results show that our system is a promising method for hand segmentation and fingertip detection which achieves a comparable performance while model complexity is suitable for real-time applications.