BusyHands: A Hand-Tool Interaction Database for Assembly Tasks Semantic Segmentation
Roy Shilkrot, Zhi Chai, Minh Hoai

TL;DR
BusyHands is a comprehensive dataset of annotated images capturing hand-tool interactions during assembly tasks, designed to advance semantic segmentation in complex, real-world scenarios involving occlusions and intricate hand poses.
Contribution
The paper introduces BusyHands, a novel large-scale dataset with pixel-level annotations for hand-tool assembly tasks, and provides benchmark evaluations of segmentation methods.
Findings
Dataset includes 7906 samples with RGB and depth images.
State-of-the-art segmentation methods evaluated on BusyHands.
Benchmark results highlight challenges in hand-tool segmentation.
Abstract
Visual segmentation has seen tremendous advancement recently with ready solutions for a wide variety of scene types, including human hands and other body parts. However, focus on segmentation of human hands while performing complex tasks, such as manual assembly, is still severely lacking. Segmenting hands from tools, work pieces, background and other body parts is extremely difficult because of self-occlusions and intricate hand grips and poses. In this paper we introduce BusyHands, a large open dataset of pixel-level annotated images of hands performing 13 different tool-based assembly tasks, from both real-world captures and virtual-world renderings. A total of 7906 samples are included in our first-in-kind dataset, with both RGB and depth images as obtained from a Kinect V2 camera and Blender. We evaluate several state-of-the-art semantic segmentation methods on our dataset as a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Human Pose and Action Recognition · Visual Attention and Saliency Detection
