Beyond Weak Perspective for Monocular 3D Human Pose Estimation

TL;DR

This paper improves monocular 3D human pose estimation by replacing weak perspective projection with full perspective projection, leading to more accurate joint and orientation predictions.

Contribution

It introduces a full perspective projection method into the SMPLify optimization, enhancing the accuracy of 3D pose and orientation estimation from monocular videos.

Findings

Full perspective projection improves estimation accuracy.

Achieved first place in 3DPW Challenge for joints orientation.

Outperforms weak perspective methods on 3DPW dataset.

Abstract

We consider the task of 3D joints location and orientation prediction from a monocular video with the skinned multi-person linear (SMPL) model. We first infer 2D joints locations with an off-the-shelf pose estimation algorithm. We use the SPIN algorithm and estimate initial predictions of body pose, shape and camera parameters from a deep regression neural network. We then adhere to the SMPLify algorithm which receives those initial parameters, and optimizes them so that inferred 3D joints from the SMPL model would fit the 2D joints locations. This algorithm involves a projection step of 3D joints to the 2D image plane. The conventional approach is to follow weak perspective assumptions which use ad-hoc focal length. Through experimentation on the 3D Poses in the Wild (3DPW) dataset, we show that using full perspective projection, with the correct camera center and an approximated focal length, provides favorable results. Our algorithm has resulted in a winning entry for the 3DPW Challenge, reaching first place in joints orientation accuracy.