Cascaded 3D Full-body Pose Regression from Single Depth Image at 100 FPS

TL;DR

This paper introduces a fast, accurate 3D full-body pose estimation method from a single depth image, operating at 100 FPS, using cascaded regressors and hierarchical kinematics to improve accuracy and preserve bone lengths.

Contribution

The novel cascaded regression approach incorporates hierarchical kinematics, enabling direct estimation of joint angles and maintaining bone length consistency, achieving state-of-the-art accuracy.

Findings

Operates at 100 frames per second.

Achieves state-of-the-art accuracy on multiple datasets.

Preserves bone length during pose estimation.

Abstract

There are increasing real-time live applications in virtual reality, where it plays an important role in capturing and retargetting 3D human pose. But it is still challenging to estimate accurate 3D pose from consumer imaging devices such as depth camera. This paper presents a novel cascaded 3D full-body pose regression method to estimate accurate pose from a single depth image at 100 fps. The key idea is to train cascaded regressors based on Gradient Boosting algorithm from pre-recorded human motion capture database. By incorporating hierarchical kinematics model of human pose into the learning procedure, we can directly estimate accurate 3D joint angles instead of joint positions. The biggest advantage of this model is that the bone length can be preserved during the whole 3D pose estimation procedure, which leads to more effective features and higher pose estimation accuracy. Our method can be used as an initialization procedure when combining with tracking methods. We demonstrate the power of our method on a wide range of synthesized human motion data from CMU mocap database, Human3.6M dataset and real human movements data captured in real time. In our comparison against previous 3D pose estimation methods and commercial system such as Kinect 2017, we achieve the state-of-the-art accuracy.