Recurrent 3D Pose Sequence Machines

TL;DR

This paper introduces a Recurrent 3D Pose Sequence Machine (RPSM) that automatically learns spatial and temporal dependencies for accurate 3D human pose estimation from monocular image sequences, outperforming existing methods.

Contribution

The proposed RPSM employs a multi-stage sequential refinement with modules for 2D pose extraction, 3D pose regression, and feature adaptation, enabling automatic learning of structural constraints and temporal context.

Findings

RPSM outperforms state-of-the-art methods on Human3.6M dataset.

RPSM achieves superior accuracy on HumanEva-I dataset.

The multi-stage framework effectively refines 3D pose predictions.

Abstract

3D human articulated pose recovery from monocular image sequences is very challenging due to the diverse appearances, viewpoints, occlusions, and also the human 3D pose is inherently ambiguous from the monocular imagery. It is thus critical to exploit rich spatial and temporal long-range dependencies among body joints for accurate 3D pose sequence prediction. Existing approaches usually manually design some elaborate prior terms and human body kinematic constraints for capturing structures, which are often insufficient to exploit all intrinsic structures and not scalable for all scenarios. In contrast, this paper presents a Recurrent 3D Pose Sequence Machine(RPSM) to automatically learn the image-dependent structural constraint and sequence-dependent temporal context by using a multi-stage sequential refinement. At each stage, our RPSM is composed of three modules to predict the 3D pose sequences based on the previously learned 2D pose representations and 3D poses: (i) a 2D pose module extracting the image-dependent pose representations, (ii) a 3D pose recurrent module regressing 3D poses and (iii) a feature adaption module serving as a bridge between module (i) and (ii) to enable the representation transformation from 2D to 3D domain. These three modules are then assembled into a sequential prediction framework to refine the predicted poses with multiple recurrent stages. Extensive evaluations on the Human3.6M dataset and HumanEva-I dataset show that our RPSM outperforms all state-of-the-art approaches for 3D pose estimation.