Toward Realistic Single-View 3D Object Reconstruction with Unsupervised Learning from Multiple Images

TL;DR

This paper introduces a new unsupervised learning algorithm for 3D object reconstruction from multiple images that overcomes symmetry limitations and enhances detail and realism, outperforming previous methods across diverse datasets.

Contribution

The paper presents a general unsupervised approach for 3D reconstruction from multiple images, removing the symmetry constraint and introducing a novel albedo loss for better detail and realism.

Findings

Outperforms previous symmetry-dependent methods in quality and robustness

Works effectively on diverse datasets including single-view, multi-view, and video data

Eliminates the symmetry requirement, broadening applicability

Abstract

Recovering the 3D structure of an object from a single image is a challenging task due to its ill-posed nature. One approach is to utilize the plentiful photos of the same object category to learn a strong 3D shape prior for the object. This approach has successfully been demonstrated by a recent work of Wu et al. (2020), which obtained impressive 3D reconstruction networks with unsupervised learning. However, their algorithm is only applicable to symmetric objects. In this paper, we eliminate the symmetry requirement with a novel unsupervised algorithm that can learn a 3D reconstruction network from a multi-image dataset. Our algorithm is more general and covers the symmetry-required scenario as a special case. Besides, we employ a novel albedo loss that improves the reconstructed details and realisticity. Our method surpasses the previous work in both quality and robustness, as shown in experiments on datasets of various structures, including single-view, multi-view, image-collection, and video sets.