3DReflecNet: A Large-Scale Dataset for 3D Reconstruction of Reflective, Transparent, and Low-Texture Objects

TL;DR

3DReflecNet is a comprehensive large-scale dataset designed to improve 3D reconstruction methods for challenging reflective, transparent, and low-texture objects, combining synthetic and real-world data for benchmarking.

Contribution

The paper introduces 3DReflecNet, a large-scale hybrid dataset with over 22 TB of data, specifically targeting the evaluation and development of 3D vision methods for difficult material surfaces.

Findings

State-of-the-art methods struggle with reflective and transparent objects.

The dataset enables benchmarking across five core 3D vision tasks.

Experiments reveal the need for more resilient 3D reconstruction models.

Abstract

Accurate 3D reconstruction of objects with reflective, transparent, or low-texture surfaces still remains notoriously challenging. Such materials often violate key assumptions in multi-view reconstruction pipelines, such as photometric consistency and the availability on distinct geometric texture cues. Existing datasets primarily focus on diffuse, textured objects, and therefore provide limited insight into performance under real-world material complexities. We introduce 3DReflecNet, a large-scale hybrid dataset exceeding 22 TB that is specifically designed to benchmark and advance 3D vision methods for these challenging materials. 3DReflecNet combines two types of data: over 120,000 synthetic instances generated via physically-based rendering of more than 12,000 shapes, and over 1,000 real-world objects captured using consumer devices. Together, these data consist of more than 7 million multi-view frames. The dataset spans diverse materials, complex lighting conditions, and a wide range of geometric forms, including shapes generated from both real and LLM-synthesized 2D images using diffusion-based pipelines. To support robust evaluation, we design benchmarks for five core tasks: image matching, structure-from-motion, novel view synthesis, reflection removal, and relighting. Extensive experiments demonstrate that state-of-the-art methods struggle to maintain accuracy across these settings, highlighting the need for more resilient 3D vision models.