Physics Informed Human Posture Estimation Based on 3D Landmarks from Monocular RGB-Videos

TL;DR

This paper introduces a real-time, physics-informed post-processing method that enhances 3D human pose estimation from monocular videos by integrating anatomical constraints, leading to more accurate and consistent results for applications like physiotherapy and sports coaching.

Contribution

It proposes a novel fusion algorithm combining BlazePose 3D and 2D estimations with biomechanical constraints, improving pose accuracy and anatomical consistency.

Findings

10.2% reduction in 3D MPJPE

16.6% decrease in angle errors

Robust, real-time performance on consumer devices

Abstract

Applications providing automated coaching for physical training are increasing in popularity, for example physical therapy. These applications rely on accurate and robust pose estimation using monocular video streams. State-of-the-art models like BlazePose excel in real-time pose tracking, but their lack of anatomical constraints indicates improvement potential by including physical knowledge. We present a real-time post-processing algorithm fusing the strengths of BlazePose 3D and 2D estimations using a weighted optimization, penalizing deviations from expected bone length and biomechanical models. Bone length estimations are refined to the individual anatomy using a Kalman filter with adapting measurement trust. Evaluation using the Physio2.2M dataset shows a 10.2 percent reduction in 3D MPJPE and a 16.6 percent decrease in errors of angles between body segments compared to BlazePose 3D estimation. Our method provides a robust, anatomically consistent pose estimation based on a computationally efficient video-to-3D pose estimation, suitable for automated physiotherapy, healthcare, and sports coaching on consumer-level laptops and mobile devices. The refinement runs on the backend with anonymized data only.