MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data
Hanwen Jiang, Zexiang Xu, Desai Xie, Ziwen Chen, Haian Jin, Fujun, Luan, Zhixin Shu, Kai Zhang, Sai Bi, Xin Sun, Jiuxiang Gu, Qixing Huang,, Georgios Pavlakos, Hao Tan

TL;DR
This paper introduces MegaSynth, a large-scale procedurally generated 3D dataset that significantly enhances 3D scene reconstruction by enabling scalable training and improving model performance across various image domains.
Contribution
MegaSynth is a novel, large-scale synthetic dataset that simplifies scene modeling to basic primitives, facilitating scalable data generation and improving 3D reconstruction models.
Findings
Training with MegaSynth improves PSNR by 1.2 to 1.8 dB.
Models trained solely on MegaSynth perform comparably to real data.
MegaSynth enhances model capability, stability, and generalization.
Abstract
We propose scaling up 3D scene reconstruction by training with synthesized data. At the core of our work is MegaSynth, a procedurally generated 3D dataset comprising 700K scenes - over 50 times larger than the prior real dataset DL3DV - dramatically scaling the training data. To enable scalable data generation, our key idea is eliminating semantic information, removing the need to model complex semantic priors such as object affordances and scene composition. Instead, we model scenes with basic spatial structures and geometry primitives, offering scalability. Besides, we control data complexity to facilitate training while loosely aligning it with real-world data distribution to benefit real-world generalization. We explore training LRMs with both MegaSynth and available real data. Experiment results show that joint training or pre-training with MegaSynth improves reconstruction quality…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Processing and 3D Reconstruction · 3D Surveying and Cultural Heritage · 3D Shape Modeling and Analysis
