Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization

TL;DR

This paper introduces Neural Vector Fields (NVF), a novel 3D shape representation combining explicit mesh manipulation and implicit distance functions, enabling resolution- and topology-agnostic surface reconstruction without surface extraction.

Contribution

The paper proposes NVF, a new shape representation that encodes both distance and direction fields via vector fields, and introduces shape codebooks and zero-curl regularization for improved reconstruction.

Findings

NVF achieves high-quality surface reconstructions across various scenarios.

Zero-curl regularization enhances the shape encoding quality.

Cross-category shape reconstruction benefits from codebook encoding.

Abstract

Recent neural networks based surface reconstruction can be roughly divided into two categories, one warping templates explicitly and the other representing 3D surfaces implicitly. To enjoy the advantages of both, we propose a novel 3D representation, Neural Vector Fields (NVF), which adopts the explicit learning process to manipulate meshes and implicit unsigned distance function (UDF) representation to break the barriers in resolution and topology. This is achieved by directly predicting the displacements from surface queries and modeling shapes as Vector Fields, rather than relying on network differentiation to obtain direction fields as most existing UDF-based methods do. In this way, our approach is capable of encoding both the distance and the direction fields so that the calculation of direction fields is differentiation-free, circumventing the non-trivial surface extraction step. Furthermore, building upon NVFs, we propose to incorporate two types of shape codebooks, \ie, NVFs (Lite or Ultra), to promote cross-category reconstruction through encoding cross-object priors. Moreover, we propose a new regularization based on analyzing the zero-curl property of NVFs, and implement this through the fully differentiable framework of our NVF (ultra). We evaluate both NVFs on four surface reconstruction scenarios, including watertight vs non-watertight shapes, category-agnostic reconstruction vs category-unseen reconstruction, category-specific, and cross-domain reconstruction.