Robust Camera-to-Mocap Calibration and Verification for Large-Scale Multi-Camera Data Capture

TL;DR

This paper introduces a robust calibration and verification system for large-scale multi-camera setups, especially fisheye cameras, ensuring accurate alignment and detecting calibration errors in real-world data collection.

Contribution

It presents a novel calibration method that handles variability and drift, along with an independent verification tool called lollypop, improving reliability in practical scenarios.

Findings

Calibration outperforms existing methods on Meta Quest 3

Lollypop reliably detects calibration degradation over time

System deployed in production data collection pipelines

Abstract

Optical motion capture (mocap) systems are widely used for ground-truth capture in AR/VR, SLAM and robotics datasets. These datasets require extrinsic calibration to align mocap coordinates to external camera frames -- a step that is subject to multiple sources of error in practice, and failures often go undetected until they corrupt downstream data. These issues are compounded for fisheye cameras, where spatially non-uniform distortion makes both calibration and verification more challenging. We present a calibration and verification system designed for this setting. Concretely, we target robustness to board-to-marker attachment variation, optimization initialization ambiguity, and session-to-session calibration drift after deployment. The calibration jointly estimates camera extrinsics and the board-to-marker transform, and uses a staged solver to improve convergence reliability under ambiguous initialization. The verification component, \lollypop, provides fast, operator-independent assessment through a measurement chain entirely independent of the calibration data. In experiments on a Meta Quest 3 headset with fisheye cameras, our calibration outperforms existing benchwork, and lollypop reliably detects calibration degradation over time. The system has been deployed in production data collection pipelines.