Egocentric Pose Estimation from Human Vision Span

TL;DR

This paper introduces a deep learning system for egocentric pose estimation from a natural human vision span, combining SLAM features and body imagery to accurately and robustly estimate head and body poses in real time for wearable devices.

Contribution

It presents a novel approach that leverages dynamic SLAM features and body imagery, explicitly enforces geometric consistency, and trains effectively with existing mocap data.

Findings

Accurate 3D head and body pose estimation in real time.

Robust system trained with existing mocap datasets.

Effective egopose estimation from a natural human visual field.

Abstract

Estimating camera wearer's body pose from an egocentric view (egopose) is a vital task in augmented and virtual reality. Existing approaches either use a narrow field of view front facing camera that barely captures the wearer, or an extruded head-mounted top-down camera for maximal wearer visibility. In this paper, we tackle the egopose estimation from a more natural human vision span, where camera wearer can be seen in the peripheral view and depending on the head pose the wearer may become invisible or has a limited partial view. This is a realistic visual field for user-centric wearable devices like glasses which have front facing wide angle cameras. Existing solutions are not appropriate for this setting, and so, we propose a novel deep learning system taking advantage of both the dynamic features from camera SLAM and the body shape imagery. We compute 3D head pose, 3D body pose, the figure/ground separation, all at the same time while explicitly enforcing a certain geometric consistency across pose attributes. We further show that this system can be trained robustly with lots of existing mocap data so we do not have to collect and annotate large new datasets. Lastly, our system estimates egopose in real time and on the fly while maintaining high accuracy.