Onboard View Planning of a Flying Camera for High Fidelity 3D Reconstruction of a Moving Actor

TL;DR

This paper introduces a drone-based motion capture system with a novel view planning method using geometric primitives and Pixel-Per-Area to achieve high-fidelity 3D reconstructions of moving actors, overcoming traditional static camera limitations.

Contribution

The work presents a complete drone-based motion capture system with a new view planning approach using PPA and geometric primitives for high-quality dynamic 3D reconstruction.

Findings

PPA correlates strongly with reconstruction quality.

Simulation results demonstrate effective view planning.

Real-world experiments confirm system produces good quality reconstructions.

Abstract

Capturing and reconstructing a human actor's motion is important for filmmaking and gaming. Currently, motion capture systems with static cameras are used for pixel-level high-fidelity reconstructions. Such setups are costly, require installation and calibration and, more importantly, confine the user to a predetermined area. In this work, we present a drone-based motion capture system that can alleviate these limitations. We present a complete system implementation and study view planning which is critical for achieving high-quality reconstructions. The main challenge for view planning for a drone-based capture system is that it needs to be performed during motion capture. To address this challenge, we introduce simple geometric primitives and show that they can be used for view planning. Specifically, we introduce Pixel-Per-Area (PPA) as a reconstruction quality proxy and plan views by maximizing the PPA of the faces of a simple geometric shape representing the actor. Through experiments in simulation, we show that PPA is highly correlated with reconstruction quality. We also conduct real-world experiments showing that our system can produce dynamic 3D reconstructions of good quality. We share our code for the simulation experiments in the link: https://github.com/Qingyuan-Jiang/view_planning_3dhuman