Mono-STAR: Mono-camera Scene-level Tracking and Reconstruction
Haonan Chang, Dhruv Metha Ramesh, Shijie Geng, Yuqiu Gan, Abdeslam, Boularias

TL;DR
Mono-STAR is a real-time 3D reconstruction system using a monocular camera that integrates semantic fusion, fast motion tracking, and topology change handling within a unified framework, advancing scene understanding.
Contribution
It introduces a novel optimization framework with optical-flow constraints and a semantic-aware deformation graph for improved scene reconstruction.
Findings
Outperforms existing state-of-the-art methods
Handles fast motion and topology changes effectively
Supports semantic fusion in real-time
Abstract
We present Mono-STAR, the first real-time 3D reconstruction system that simultaneously supports semantic fusion, fast motion tracking, non-rigid object deformation, and topological change under a unified framework. The proposed system solves a new optimization problem incorporating optical-flow-based 2D constraints to deal with fast motion and a novel semantic-aware deformation graph (SAD-graph) for handling topology change. We test the proposed system under various challenging scenes and demonstrate that it significantly outperforms existing state-of-the-art methods.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Vision and Imaging · Robotics and Sensor-Based Localization · Advanced Image and Video Retrieval Techniques
MethodsTest
