NeAT: Learning Neural Implicit Surfaces with Arbitrary Topologies from Multi-view Images

TL;DR

NeAT is a neural rendering framework that reconstructs 3D surfaces with arbitrary topologies from multi-view images, overcoming the limitations of previous methods restricted to closed surfaces.

Contribution

NeAT introduces a novel level set representation with a validity branch and a new neural volume rendering method for open and complex surfaces.

Findings

Outperforms state-of-the-art in open surface reconstruction

Faithfully reconstructs both watertight and non-watertight surfaces

Supports easy conversion to mesh with Marching Cubes

Abstract

Recent progress in neural implicit functions has set new state-of-the-art in reconstructing high-fidelity 3D shapes from a collection of images. However, these approaches are limited to closed surfaces as they require the surface to be represented by a signed distance field. In this paper, we propose NeAT, a new neural rendering framework that can learn implicit surfaces with arbitrary topologies from multi-view images. In particular, NeAT represents the 3D surface as a level set of a signed distance function (SDF) with a validity branch for estimating the surface existence probability at the query positions. We also develop a novel neural volume rendering method, which uses SDF and validity to calculate the volume opacity and avoids rendering points with low validity. NeAT supports easy field-to-mesh conversion using the classic Marching Cubes algorithm. Extensive experiments on DTU, MGN, and Deep Fashion 3D datasets indicate that our approach is able to faithfully reconstruct both watertight and non-watertight surfaces. In particular, NeAT significantly outperforms the state-of-the-art methods in the task of open surface reconstruction both quantitatively and qualitatively.