A Simple and Scalable Shape Representation for 3D Reconstruction

TL;DR

This paper introduces a simple, scalable 3D shape representation method using PCA on signed distance functions, enabling high-quality reconstructions without complex decoders, and demonstrating competitive results.

Contribution

It shows that a linear PCA-based decoder suffices for high-quality 3D reconstruction, simplifying models and improving scalability compared to complex architectures.

Findings

Achieves high-quality 3D reconstructions with a linear PCA decoder.

Scales effectively to higher resolutions than voxel-based methods.

Fine-tuning on target tasks improves reconstruction quality.

Abstract

Deep learning applied to the reconstruction of 3D shapes has seen growing interest. A popular approach to 3D reconstruction and generation in recent years has been the CNN encoder-decoder model usually applied in voxel space. However, this often scales very poorly with the resolution limiting the effectiveness of these models. Several sophisticated alternatives for decoding to 3D shapes have been proposed typically relying on complex deep learning architectures for the decoder model. In this work, we show that this additional complexity is not necessary, and that we can actually obtain high quality 3D reconstruction using a linear decoder, obtained from principal component analysis on the signed distance function (SDF) of the surface. This approach allows easily scaling to larger resolutions. We show in multiple experiments that our approach is competitive with state-of-the-art methods. It also allows the decoder to be fine-tuned on the target task using a loss designed specifically for SDF transforms, obtaining further gains.