End-to-End Motion Capture from Rigid Body Markers with Geodesic Loss

TL;DR

This paper introduces a new rigid body marker system and a deep learning model with geodesic loss for accurate, real-time motion capture that simplifies setup and reduces computation compared to traditional methods.

Contribution

The paper presents the Rigid Body Marker (RBM) as a novel fundamental unit for motion capture and develops an end-to-end deep learning approach with geodesic loss for efficient pose estimation.

Findings

Achieves state-of-the-art accuracy in body pose estimation.

Requires significantly less computation than optimization-based methods.

Demonstrates practical viability with real-world data.

Abstract

Marker-based optical motion capture (MoCap), while long regarded as the gold standard for accuracy, faces practical challenges, such as time-consuming preparation and marker identification ambiguity, due to its reliance on dense marker configurations, which fundamentally limit its scalability. To address this, we introduce a novel fundamental unit for MoCap, the Rigid Body Marker (RBM), which provides unambiguous 6-DoF data and drastically simplifies setup. Leveraging this new data modality, we develop a deep-learning-based regression model that directly estimates SMPL parameters under a geodesic loss. This end-to-end approach matches the performance of optimization-based methods while requiring over an order of magnitude less computation. Trained on synthesized data from the AMASS dataset, our end-to-end model achieves state-of-the-art accuracy in body pose estimation. Real-world data captured using a Vicon optical tracking system further demonstrates the practical viability of our approach. Overall, the results show that combining sparse 6-DoF RBM with a manifold-aware geodesic loss yields a practical and high-fidelity solution for real-time MoCap in graphics, virtual reality, and biomechanics.