QuaMo: Quaternion Motions for Vision-based 3D Human Kinematics Capture

TL;DR

QuaMo introduces a quaternion differential equations approach for vision-based 3D human motion capture, effectively addressing discontinuities and improving accuracy over traditional methods, especially in online settings.

Contribution

The paper proposes a novel quaternion-based method using QDE and a state-space model with acceleration enhancement for more stable and accurate 3D human kinematics estimation.

Findings

Outperforms state-of-the-art methods on multiple datasets.

Produces continuous, stable motion reconstructions without discontinuities.

Accurately estimates 3D human kinematics in real-time.

Abstract

Vision-based 3D human motion capture from videos remains a challenge in computer vision. Traditional 3D pose estimation approaches often ignore the temporal consistency between frames, causing implausible and jittery motion. The emerging field of kinematics-based 3D motion capture addresses these issues by estimating the temporal transitioning between poses instead. A major drawback in current kinematics approaches is their reliance on Euler angles. Despite their simplicity, Euler angles suffer from discontinuity that leads to unstable motion reconstructions, especially in online settings where trajectory refinement is unavailable. Contrarily, quaternions have no discontinuity and can produce continuous transitions between poses. In this paper, we propose QuaMo, a novel Quaternion Motions method using quaternion differential equations (QDE) for human kinematics capture. We utilize the state-space model, an effective system for describing real-time kinematics estimations, with quaternion state and the QDE describing quaternion velocity. The corresponding angular acceleration is computed from a meta-PD controller with a novel acceleration enhancement that adaptively regulates the control signals as the human quickly changes to a new pose. Unlike previous work, our QDE is solved under the quaternion unit-sphere constraint that results in more accurate estimations. Experimental results show that our novel formulation of the QDE with acceleration enhancement accurately estimates 3D human kinematics with no discontinuity and minimal implausibilities. QuaMo outperforms comparable state-of-the-art methods on multiple datasets, namely Human3.6M, Fit3D, SportsPose and AIST. The code is available at https://github.com/cuongle1206/QuaMo