SimNP: Learning Self-Similarity Priors Between Neural Points

TL;DR

SimNP introduces a novel neural point representation that captures category-level self-similarities, enabling detailed 3D object reconstruction and inference of unobserved regions by leveraging learned point correlations.

Contribution

It is the first to combine neural point radiance fields with category-level self-similarity, improving detail and unobserved region inference in 3D reconstruction.

Findings

Outperforms previous methods in reconstructing symmetric unseen regions.

Provides semantic correspondences between object instances.

Stores high detail in locally supported object regions.

Abstract

Existing neural field representations for 3D object reconstruction either (1) utilize object-level representations, but suffer from low-quality details due to conditioning on a global latent code, or (2) are able to perfectly reconstruct the observations, but fail to utilize object-level prior knowledge to infer unobserved regions. We present SimNP, a method to learn category-level self-similarities, which combines the advantages of both worlds by connecting neural point radiance fields with a category-level self-similarity representation. Our contribution is two-fold. (1) We design the first neural point representation on a category level by utilizing the concept of coherent point clouds. The resulting neural point radiance fields store a high level of detail for locally supported object regions. (2) We learn how information is shared between neural points in an unconstrained and unsupervised fashion, which allows to derive unobserved regions of an object during the reconstruction process from given observations. We show that SimNP is able to outperform previous methods in reconstructing symmetric unseen object regions, surpassing methods that build upon category-level or pixel-aligned radiance fields, while providing semantic correspondences between instances