Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras

TL;DR

This paper proposes a novel method for dynamic multi-person mesh recovery from uncalibrated multi-view cameras by integrating motion priors and physics-geometry consistency, enabling accurate 3D human motion capture despite noisy inputs.

Contribution

It introduces a unified optimization framework that jointly estimates camera parameters and human meshes using motion priors and physics-geometry constraints from uncalibrated views.

Findings

Achieves accurate camera and human motion estimation with noisy data

Uses physics-geometry consistency to reduce noise in human semantics

Demonstrates effectiveness through experimental validation

Abstract

Dynamic multi-person mesh recovery has been a hot topic in 3D vision recently. However, few works focus on the multi-person motion capture from uncalibrated cameras, which mainly faces two challenges: the one is that inter-person interactions and occlusions introduce inherent ambiguities for both camera calibration and motion capture; The other is that a lack of dense correspondences can be used to constrain sparse camera geometries in a dynamic multi-person scene. Our key idea is incorporating motion prior knowledge into simultaneous optimization of extrinsic camera parameters and human meshes from noisy human semantics. First, we introduce a physics-geometry consistency to reduce the low and high frequency noises of the detected human semantics. Then a novel latent motion prior is proposed to simultaneously optimize extrinsic camera parameters and coherent human motions from slightly noisy inputs. Experimental results show that accurate camera parameters and human motions can be obtained through one-stage optimization. The codes will be publicly available at~\url{https://www.yangangwang.com}.