Spatiotemporal Multi-Camera Calibration using Freely Moving People

TL;DR

This paper introduces a unified, marker-free method for calibrating multiple moving cameras by leveraging human motion and 3D pose estimation, improving accuracy in dynamic multi-view scenes.

Contribution

It presents a novel probabilistic framework that jointly estimates camera parameters, temporal offsets, and person associations using 3D human poses from monocular estimates.

Findings

Effective calibration on synthetic and real data

Joint spatiotemporal and correspondence estimation improves accuracy

Flexible and practical for dynamic multi-person scenes

Abstract

We propose a novel method for spatiotemporal multi-camera calibration using freely moving people in multiview videos. Since calibrating multiple cameras and finding matches across their views are inherently interdependent, performing both in a unified framework poses a significant challenge. We address these issues as a single registration problem of matching two sets of 3D points, leveraging human motion in dynamic multi-person scenes. To this end, we utilize 3D human poses obtained from an off-the-shelf monocular 3D human pose estimator and transform them into 3D points on a unit sphere, to solve the rotation, time offset, and the association alternatingly. We employ a probabilistic approach that can jointly solve both problems of aligning spatiotemporal data and establishing correspondences through soft assignment between two views. The translation is determined by applying coplanarity constraints. The pairwise registration results are integrated into a multiview setup, and then a nonlinear optimization method is used to improve the accuracy of the camera poses, temporal offsets, and multi-person associations. Extensive experiments on synthetic and real data demonstrate the effectiveness and flexibility of the proposed method as a practical marker-free calibration tool.