D$^2$IM-Net: Learning Detail Disentangled Implicit Fields from Single Images

TL;DR

D$^2$IM-Net is a novel single-view 3D reconstruction network that disentangles coarse shape and surface details by learning implicit fields and displacement maps, improving detail recovery from images.

Contribution

It introduces a dual-decoder approach that separately reconstructs coarse shapes and surface details, with a novel Laplacian loss for enhanced detail preservation.

Findings

Successfully reconstructs detailed 3D shapes from single images.

Outperforms existing methods in detail accuracy and shape fidelity.

Employs a novel loss function to improve surface detail recovery.

Abstract

We present the first single-view 3D reconstruction network aimed at recovering geometric details from an input image which encompass both topological shape structures and surface features. Our key idea is to train the network to learn a detail disentangled reconstruction consisting of two functions, one implicit field representing the coarse 3D shape and the other capturing the details. Given an input image, our network, coined D$^2$IM-Net, encodes it into global and local features which are respectively fed into two decoders. The base decoder uses the global features to reconstruct a coarse implicit field, while the detail decoder reconstructs, from the local features, two displacement maps, defined over the front and back sides of the captured object. The final 3D reconstruction is a fusion between the base shape and the displacement maps, with three losses enforcing the recovery of coarse shape, overall structure, and surface details via a novel Laplacian term.