3D Reconstruction of Sculptures from Single Images via Unsupervised Domain Adaptation on Implicit Models

TL;DR

This paper presents an unsupervised domain adaptation approach for 3D reconstruction of sculptures from single images, leveraging models trained on human data to improve sculpture reconstruction in VR applications.

Contribution

It introduces a novel unsupervised domain adaptation method for 3D implicit reconstruction models from human to sculpture domains, addressing data scarcity and domain shift issues.

Findings

The proposed method outperforms existing approaches in shape quality.

Ablation studies validate the effectiveness of each component.

User studies confirm improved reconstruction realism.

Abstract

Acquiring the virtual equivalent of exhibits, such as sculptures, in virtual reality (VR) museums, can be labour-intensive and sometimes infeasible. Deep learning based 3D reconstruction approaches allow us to recover 3D shapes from 2D observations, among which single-view-based approaches can reduce the need for human intervention and specialised equipment in acquiring 3D sculptures for VR museums. However, there exist two challenges when attempting to use the well-researched human reconstruction methods: limited data availability and domain shift. Considering sculptures are usually related to humans, we propose our unsupervised 3D domain adaptation method for adapting a single-view 3D implicit reconstruction model from the source (real-world humans) to the target (sculptures) domain. We have compared the generated shapes with other methods and conducted ablation studies as well as a user study to demonstrate the effectiveness of our adaptation method. We also deploy our results in a VR application.