GridFormer: Point-Grid Transformer for Surface Reconstruction

TL;DR

GridFormer introduces a novel point-grid transformer that enhances 3D surface reconstruction by effectively combining grid and point features, resulting in more precise geometry with improved efficiency.

Contribution

It proposes a new attention mechanism between grid and point features, and a boundary optimization strategy, advancing the accuracy and efficiency of surface reconstruction from point clouds.

Findings

Outperforms state-of-the-art methods on benchmark datasets.

Produces more accurate and detailed surface reconstructions.

Efficiently balances detail preservation and computational cost.

Abstract

Implicit neural networks have emerged as a crucial technology in 3D surface reconstruction. To reconstruct continuous surfaces from discrete point clouds, encoding the input points into regular grid features (plane or volume) has been commonly employed in existing approaches. However, these methods typically use the grid as an index for uniformly scattering point features. Compared with the irregular point features, the regular grid features may sacrifice some reconstruction details but improve efficiency. To take full advantage of these two types of features, we introduce a novel and high-efficiency attention mechanism between the grid and point features named Point-Grid Transformer (GridFormer). This mechanism treats the grid as a transfer point connecting the space and point cloud. Our method maximizes the spatial expressiveness of grid features and maintains computational efficiency. Furthermore, optimizing predictions over the entire space could potentially result in blurred boundaries. To address this issue, we further propose a boundary optimization strategy incorporating margin binary cross-entropy loss and boundary sampling. This approach enables us to achieve a more precise representation of the object structure. Our experiments validate that our method is effective and outperforms the state-of-the-art approaches under widely used benchmarks by producing more precise geometry reconstructions. The code is available at https://github.com/list17/GridFormer.