Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling

TL;DR

This paper introduces a novel unsupervised approach for 3D human pose estimation that models pose deformation using non-rigid structure-from-motion and diffusion-based priors, outperforming existing methods.

Contribution

It proposes a new modeling framework combining NRSfM and diffusion priors for unsupervised 3D pose estimation, addressing pose deformation modeling.

Findings

Outperforms state-of-the-art on mainstream datasets

Effectively models pose deformation in motion

Uses diffusion-based prior to improve learning

Abstract

Most of the previous 3D human pose estimation work relied on the powerful memory capability of the network to obtain suitable 2D-3D mappings from the training data. Few works have studied the modeling of human posture deformation in motion. In this paper, we propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton, and a frame-by-frame skeleton deformation. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence, and then sum them to obtain the pose of each frame. Subsequently, a loss term based on the diffusion model is used to ensure that the pipeline learns the correct prior motion knowledge. Finally, we have evaluated our proposed method on mainstream datasets and obtained superior results outperforming the state-of-the-art.