CasCalib: Cascaded Calibration for Motion Capture from Sparse Unsynchronized Cameras

TL;DR

CasCalib is an automated, robust motion capture system that calibrates multiple unsynchronized cameras using scene persons, eliminating manual calibration and synchronization for accurate 3D human pose estimation.

Contribution

It introduces a cascaded optimization approach for fully automatic multi-view camera calibration and synchronization using scene persons, handling multiple individuals and diverse scenarios.

Findings

Successfully calibrates and synchronizes multi-view cameras automatically.

Demonstrates robustness across diverse multi-view benchmarks.

Provides an easy-to-use, flexible motion capture toolbox.

Abstract

It is now possible to estimate 3D human pose from monocular images with off-the-shelf 3D pose estimators. However, many practical applications require fine-grained absolute pose information for which multi-view cues and camera calibration are necessary. Such multi-view recordings are laborious because they require manual calibration, and are expensive when using dedicated hardware. Our goal is full automation, which includes temporal synchronization, as well as intrinsic and extrinsic camera calibration. This is done by using persons in the scene as the calibration objects. Existing methods either address only synchronization or calibration, assume one of the former as input, or have significant limitations. A common limitation is that they only consider single persons, which eases correspondence finding. We attain this generality by partitioning the high-dimensional time and calibration space into a cascade of subspaces and introduce tailored algorithms to optimize each efficiently and robustly. The outcome is an easy-to-use, flexible, and robust motion capture toolbox that we release to enable scientific applications, which we demonstrate on diverse multi-view benchmarks. Project website: https://github.com/jamestang1998/CasCalib.