Ladybird: Quasi-Monte Carlo Sampling for Deep Implicit Field Based 3D Reconstruction with Symmetry

TL;DR

Ladybird introduces a novel sampling scheme based on Farthest Point Sampling and a symmetry-based feature fusion method, significantly improving 3D reconstruction quality from single images using deep implicit fields.

Contribution

The paper proposes a theoretically motivated sampling scheme and a symmetry-aware feature fusion technique, enhancing deep implicit 3D reconstruction from single images.

Findings

Improved convergence speed in training with the new sampling scheme.

High-quality 3D reconstructions demonstrated on ShapeNet dataset.

Competitive results in Chamfer distance, Earth Mover's distance, and IoU.

Abstract

Deep implicit field regression methods are effective for 3D reconstruction from single-view images. However, the impact of different sampling patterns on the reconstruction quality is not well-understood. In this work, we first study the effect of point set discrepancy on the network training. Based on Farthest Point Sampling algorithm, we propose a sampling scheme that theoretically encourages better generalization performance, and results in fast convergence for SGD-based optimization algorithms. Secondly, based on the reflective symmetry of an object, we propose a feature fusion method that alleviates issues due to self-occlusions which makes it difficult to utilize local image features. Our proposed system Ladybird is able to create high quality 3D object reconstructions from a single input image. We evaluate Ladybird on a large scale 3D dataset (ShapeNet) demonstrating highly competitive results in terms of Chamfer distance, Earth Mover's distance and Intersection Over Union (IoU).