BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos

TL;DR

BioPose is a learning-based framework that predicts biomechanically accurate 3D human poses from monocular videos, overcoming limitations of parametric models and costly motion capture systems.

Contribution

It introduces a novel multi-component framework combining mesh recovery, inverse kinematics, and 2D refinement for accurate biomechanical pose estimation from monocular videos.

Findings

Outperforms state-of-the-art methods on benchmark datasets

Achieves high accuracy in joint localization and movement capture

Demonstrates applicability in biomechanics, healthcare, and robotics

Abstract

Recent advancements in 3D human pose estimation from single-camera images and videos have relied on parametric models, like SMPL. However, these models oversimplify anatomical structures, limiting their accuracy in capturing true joint locations and movements, which reduces their applicability in biomechanics, healthcare, and robotics. Biomechanically accurate pose estimation, on the other hand, typically requires costly marker-based motion capture systems and optimization techniques in specialized labs. To bridge this gap, we propose BioPose, a novel learning-based framework for predicting biomechanically accurate 3D human pose directly from monocular videos. BioPose includes three key components: a Multi-Query Human Mesh Recovery model (MQ-HMR), a Neural Inverse Kinematics (NeurIK) model, and a 2D-informed pose refinement technique. MQ-HMR leverages a multi-query deformable transformer to extract multi-scale fine-grained image features, enabling precise human mesh recovery. NeurIK treats the mesh vertices as virtual markers, applying a spatial-temporal network to regress biomechanically accurate 3D poses under anatomical constraints. To further improve 3D pose estimations, a 2D-informed refinement step optimizes the query tokens during inference by aligning the 3D structure with 2D pose observations. Experiments on benchmark datasets demonstrate that BioPose significantly outperforms state-of-the-art methods. Project website: \url{https://m-usamasaleem.github.io/publication/BioPose/BioPose.html}.