D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning

TL;DR

This paper introduces D-OccNet, a novel cross-domain learning approach that converts 2D images into point clouds to improve detailed 3D reconstruction quality over existing occupancy network methods.

Contribution

The paper proposes a new method that leverages cross-domain learning between images and point clouds, enhancing 3D reconstruction detail and quality.

Findings

D-OccNet outperforms existing occupancy networks in visual quality.

Converting 2D images to point clouds improves 3D surface reconstruction.

The approach captures more detailed 3D structures.

Abstract

Deep learning based 3D reconstruction of single view 2D image is becoming increasingly popular due to their wide range of real-world applications, but this task is inherently challenging because of the partial observability of an object from a single perspective. Recently, state of the art probability based Occupancy Networks reconstructed 3D surfaces from three different types of input domains: single view 2D image, point cloud and voxel. In this study, we extend the work on Occupancy Networks by exploiting cross-domain learning of image and point cloud domains. Specifically, we first convert the single view 2D image into a simpler point cloud representation, and then reconstruct a 3D surface from it. Our network, the Double Occupancy Network (D-OccNet) outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.