Estimating 3D Camera Pose from 2D Pedestrian Trajectories

TL;DR

This paper introduces a neural network approach that estimates 3D camera pose from 2D pedestrian trajectories, trained solely on synthetic data, enabling accurate re-calibration without real-world annotations.

Contribution

The paper presents a novel data-driven method using synthetic training data to infer camera pose from pedestrian trajectories, bypassing the need for 3D point annotations.

Findings

Achieves approximately 50% improvement in position accuracy.

Achieves approximately 50% improvement in orientation accuracy.

Effective across multiple real-world scenes without real data training.

Abstract

We consider the task of re-calibrating the 3D pose of a static surveillance camera, whose pose may change due to external forces, such as birds, wind, falling objects or earthquakes. Conventionally, camera pose estimation can be solved with a PnP (Perspective-n-Point) method using 2D-to-3D feature correspondences, when 3D points are known. However, 3D point annotations are not always available or practical to obtain in real-world applications. We propose an alternative strategy for extracting 3D information to solve for camera pose by using pedestrian trajectories. We observe that 2D pedestrian trajectories indirectly contain useful 3D information that can be used for inferring camera pose. To leverage this information, we propose a data-driven approach by training a neural network (NN) regressor to model a direct mapping from 2D pedestrian trajectories projected on the image plane to 3D camera pose. We demonstrate that our regressor trained only on synthetic data can be directly applied to real data, thus eliminating the need to label any real data. We evaluate our method across six different scenes from the Town Centre Street and DUKEMTMC datasets. Our method achieves an improvement of $\sim50\%$ on both position and orientation prediction accuracy when compared to other SOTA methods.