RayletDF: Raylet Distance Fields for Generalizable 3D Surface Reconstruction from Point Clouds or Gaussians

TL;DR

RayletDF is a novel 3D surface reconstruction method that efficiently predicts surface points from point clouds or Gaussians using a new raylet distance field technique, demonstrating strong generalization across datasets.

Contribution

Introduces RayletDF, a generalizable approach with a raylet distance field for direct surface prediction from raw point clouds or Gaussians, improving efficiency and accuracy.

Findings

Outperforms existing methods on multiple datasets.

Achieves real-time surface reconstruction in a single forward pass.

Demonstrates strong generalization to unseen data.

Abstract

In this paper, we present a generalizable method for 3D surface reconstruction from raw point clouds or pre-estimated 3D Gaussians by 3DGS from RGB images. Unlike existing coordinate-based methods which are often computationally intensive when rendering explicit surfaces, our proposed method, named RayletDF, introduces a new technique called raylet distance field, which aims to directly predict surface points from query rays. Our pipeline consists of three key modules: a raylet feature extractor, a raylet distance field predictor, and a multi-raylet blender. These components work together to extract fine-grained local geometric features, predict raylet distances, and aggregate multiple predictions to reconstruct precise surface points. We extensively evaluate our method on multiple public real-world datasets, demonstrating superior performance in surface reconstruction from point clouds or 3D Gaussians. Most notably, our method achieves exceptional generalization ability, successfully recovering 3D surfaces in a single-forward pass across unseen datasets in testing.