Gesture-based Bootstrapping for Egocentric Hand Segmentation
Yubo Zhang, Vishnu Naresh Boddeti, Kris M. Kitani

TL;DR
This paper presents a person-specific hand segmentation method for egocentric images that uses gesture-based bootstrapping and deep learning, eliminating the need for manual labeling and adapting to individual users and environments.
Contribution
It introduces an interactive, data-free approach combining gesture and appearance neural networks for personalized hand segmentation in wearable camera images.
Findings
Achieved over 0.8 F1 score across diverse datasets.
Improved baseline performance by over 10%.
Robust to variations in illumination and hand appearance.
Abstract
Accurately identifying hands in images is a key sub-task for human activity understanding with wearable first-person point-of-view cameras. Traditional hand segmentation approaches rely on a large corpus of manually labeled data to generate robust hand detectors. However, these approaches still face challenges as the appearance of the hand varies greatly across users, tasks, environments or illumination conditions. A key observation in the case of many wearable applications and interfaces is that, it is only necessary to accurately detect the user's hands in a specific situational context. Based on this observation, we introduce an interactive approach to learn a person-specific hand segmentation model that does not require any manually labeled training data. Our approach proceeds in two steps, an interactive bootstrapping step for identifying moving hand regions, followed by learning a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Human Pose and Action Recognition · Robot Manipulation and Learning
