Vis2Mesh: Efficient Mesh Reconstruction from Unstructured Point Clouds of Large Scenes with Learned Virtual View Visibility

TL;DR

This paper introduces Vis2Mesh, a framework that reconstructs 3D meshes from unstructured point clouds by leveraging learned visibility in virtual views, combining depth completion and graph-cut optimization for large scene efficiency.

Contribution

It proposes a novel visibility prediction network and an adaptive weighting scheme, enabling efficient and scalable mesh reconstruction from large point clouds with improved accuracy.

Findings

Outperforms state-of-the-art methods on large scenes

Demonstrates high transferability and robustness

Efficiently handles large point clouds with a simple binary classification task

Abstract

We present a novel framework for mesh reconstruction from unstructured point clouds by taking advantage of the learned visibility of the 3D points in the virtual views and traditional graph-cut based mesh generation. Specifically, we first propose a three-step network that explicitly employs depth completion for visibility prediction. Then the visibility information of multiple views is aggregated to generate a 3D mesh model by solving an optimization problem considering visibility in which a novel adaptive visibility weighting in surface determination is also introduced to suppress line of sight with a large incident angle. Compared to other learning-based approaches, our pipeline only exercises the learning on a 2D binary classification task, \ie, points visible or not in a view, which is much more generalizable and practically more efficient and capable to deal with a large number of points. Experiments demonstrate that our method with favorable transferability and robustness, and achieve competing performances \wrt state-of-the-art learning-based approaches on small complex objects and outperforms on large indoor and outdoor scenes. Code is available at https://github.com/GDAOSU/vis2mesh.