OpenMaterial: A Large-scale Dataset of Complex Materials for 3D Reconstruction

TL;DR

OpenMaterial is a comprehensive large-scale dataset designed to benchmark 3D reconstruction methods on complex materials, addressing challenges posed by optical properties like transparency and reflections in multi-view imaging.

Contribution

It introduces the first extensive benchmark dataset with diverse materials and lighting conditions, enabling evaluation of material-aware 3D reconstruction methods.

Findings

11 state-of-the-art methods evaluated

Material type significantly affects reconstruction quality

OpenMaterial facilitates development of more robust 3D techniques

Abstract

Recent advances in deep learning, such as neural radiance fields and implicit neural representations, have significantly advanced 3D reconstruction. However, accurately reconstructing objects with complex optical properties, such as metals, glass, and plastics, remains challenging due to the breakdown of multi-view color consistency in the presence of specular reflections, refractions, and transparency. This limitation is further exacerbated by the lack of benchmark datasets that explicitly model material-dependent light transport. To address this, we introduce OpenMaterial, a large-scale semi-synthetic dataset for benchmarking material-aware 3D reconstruction. It comprises 1,001 objects spanning 295 distinct materials, including conductors, dielectrics, plastics, and their roughened variants, captured under 714 diverse lighting conditions. By integrating lab-measured Index of Refraction (IOR) spectra, OpenMaterial enables the generation of high-fidelity multi-view images that accurately simulate complex light-matter interactions. It provides multi-view images, 3D shape models, camera poses, depth maps, and object masks, establishing the first extensive benchmark for evaluating 3D reconstruction on challenging materials. We evaluate 11 state-of-the-art methods for 3D reconstruction and novel view synthesis, conducting ablation studies to assess the impact of material type, shape complexity, and illumination on reconstruction performance. Our results indicate that OpenMaterial provides a strong and fair basis for developing more robust, physically-informed 3D reconstruction techniques to better handle real-world optical complexities.