Free-Moving Object Reconstruction and Pose Estimation with Virtual Camera
Haixin Shi, Yinlin Hu, Daniel Koguciuk, Juan-Ting Lin, Mathieu, Salzmann, David Ferstl

TL;DR
This paper introduces a novel method for reconstructing and estimating the pose of free-moving objects from monocular RGB videos without prior assumptions, using a virtual camera system to optimize shape and pose globally.
Contribution
The method enables free interaction with objects in front of a moving camera without prior knowledge, utilizing a virtual camera to improve optimization efficiency.
Findings
Outperforms most existing methods on standard datasets.
Achieves comparable results to prior-based techniques.
Effectively reconstructs object shape and pose in egocentric videos.
Abstract
We propose an approach for reconstructing free-moving object from a monocular RGB video. Most existing methods either assume scene prior, hand pose prior, object category pose prior, or rely on local optimization with multiple sequence segments. We propose a method that allows free interaction with the object in front of a moving camera without relying on any prior, and optimizes the sequence globally without any segments. We progressively optimize the object shape and pose simultaneously based on an implicit neural representation. A key aspect of our method is a virtual camera system that reduces the search space of the optimization significantly. We evaluate our method on the standard HO3D dataset and a collection of egocentric RGB sequences captured with a head-mounted device. We demonstrate that our approach outperforms most methods significantly, and is on par with recent…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Image and Object Detection Techniques
