Domain Adaptation for Real-World Single View 3D Reconstruction

TL;DR

This paper explores domain adaptation techniques to improve single view 3D reconstruction from synthetic to real-world images, proposing a new architecture and demonstrating results on the ODDS dataset.

Contribution

The paper introduces a novel architecture leveraging class labels for unsupervised domain adaptation in 3D reconstruction, outperforming existing methods on real-world datasets.

Findings

Domain adaptation improves 3D reconstruction accuracy on real-world data.

The proposed architecture outperforms baseline methods on ODDS dataset.

First application of multiview reconstruction techniques on ODDS dataset.

Abstract

Deep learning-based object reconstruction algorithms have shown remarkable improvements over classical methods. However, supervised learning based methods perform poorly when the training data and the test data have different distributions. Indeed, most current works perform satisfactorily on the synthetic ShapeNet dataset, but dramatically fail in when presented with real world images. To address this issue, unsupervised domain adaptation can be used transfer knowledge from the labeled synthetic source domain and learn a classifier for the unlabeled real target domain. To tackle this challenge of single view 3D reconstruction in the real domain, we experiment with a variety of domain adaptation techniques inspired by the maximum mean discrepancy (MMD) loss, Deep CORAL, and the domain adversarial neural network (DANN). From these findings, we additionally propose a novel architecture which takes advantage of the fact that in this setting, target domain data is unsupervised with regards to the 3D model but supervised for class labels. We base our framework off a recent network called pix2vox. Results are performed with ShapeNet as the source domain and domains within the Object Dataset Domain Suite (ODDS) dataset as the target, which is a real world multiview, multidomain image dataset. The domains in ODDS vary in difficulty, allowing us to assess notions of domain gap size. Our results are the first in the multiview reconstruction literature using this dataset.