Consensus-based Optimization for 3D Human Pose Estimation in Camera Coordinates

TL;DR

This paper introduces a novel approach for 3D human pose estimation in camera coordinates, combining multi-view data and a consensus-based optimization algorithm to improve accuracy and generalization across different camera settings.

Contribution

It proposes a view frustum space formulation and a consensus-based optimization method for multi-view pose estimation from uncalibrated images, with a single monocular training process.

Findings

Reduces prediction error by 32% on standard datasets.

Achieves 80mm average absolute pose error for monocular estimation.

Achieves 51mm average error for multi-view estimation.

Abstract

3D human pose estimation is frequently seen as the task of estimating 3D poses relative to the root body joint. Alternatively, we propose a 3D human pose estimation method in camera coordinates, which allows effective combination of 2D annotated data and 3D poses and a straightforward multi-view generalization. To that end, we cast the problem as a view frustum space pose estimation, where absolute depth prediction and joint relative depth estimations are disentangled. Final 3D predictions are obtained in camera coordinates by the inverse camera projection. Based on this, we also present a consensus-based optimization algorithm for multi-view predictions from uncalibrated images, which requires a single monocular training procedure. Although our method is indirectly tied to the training camera intrinsics, it still converges for cameras with different intrinsic parameters, resulting in coherent estimations up to a scale factor. Our method improves the state of the art on well known 3D human pose datasets, reducing the prediction error by 32% in the most common benchmark. We also reported our results in absolute pose position error, achieving 80~mm for monocular estimations and 51~mm for multi-view, on average.