Hand Action Detection from Ego-centric Depth Sequences with Error-correcting Hough Transform

TL;DR

This paper introduces a novel Hough transform-based method with an error-correcting component for detecting hand actions in ego-centric depth videos, addressing challenges from complex hand movements and camera motion.

Contribution

It proposes a discriminatively learned error-correcting Hough transform approach and provides a new annotated dataset for hand action detection from ego-centric depth sequences.

Findings

System achieves satisfactory results on real-life dataset

New dataset with 3,177 annotated action subsequences

Method effectively handles complex hand articulations and camera motion

Abstract

Detecting hand actions from ego-centric depth sequences is a practically challenging problem, owing mostly to the complex and dexterous nature of hand articulations as well as non-stationary camera motion. We address this problem via a Hough transform based approach coupled with a discriminatively learned error-correcting component to tackle the well known issue of incorrect votes from the Hough transform. In this framework, local parts vote collectively for the start $\&$ end positions of each action over time. We also construct an in-house annotated dataset of 300 long videos, containing 3,177 single-action subsequences over 16 action classes collected from 26 individuals. Our system is empirically evaluated on this real-life dataset for both the action recognition and detection tasks, and is shown to produce satisfactory results. To facilitate reproduction, the new dataset and our implementation are also provided online.