Recognition from Hand Cameras
Cheng-Sheng Chan, Shou-Zhong Chen, Pei-Xuan Xie, Chiung-Chih Chang,, Min Sun

TL;DR
This paper introduces a wrist-mounted HandCam system for recognizing hand activities, demonstrating its advantages over HeadCam and proposing a novel deep learning approach that combines both camera views for improved accuracy.
Contribution
The paper presents a new HandCam dataset, a deep learning method for hand activity recognition, and a two-stream network combining HandCam and HeadCam data, advancing hand activity recognition technology.
Findings
HandCam outperforms HeadCam in recognition tasks.
Aligning videos from different users improves accuracy.
Two-stream network achieves best performance in most tasks.
Abstract
We revisit the study of a wrist-mounted camera system (referred to as HandCam) for recognizing activities of hands. HandCam has two unique properties as compared to egocentric systems (referred to as HeadCam): (1) it avoids the need to detect hands; (2) it more consistently observes the activities of hands. By taking advantage of these properties, we propose a deep-learning-based method to recognize hand states (free v.s. active hands, hand gestures, object categories), and discover object categories. Moreover, we propose a novel two-streams deep network to further take advantage of both HandCam and HeadCam. We have collected a new synchronized HandCam and HeadCam dataset with 20 videos captured in three scenes for hand states recognition. Experiments show that our HandCam system consistently outperforms a deep-learning-based HeadCam method (with estimated manipulation regions) and a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Human Pose and Action Recognition · Gaze Tracking and Assistive Technology
