Sparse Point Cloud Patches Rendering via Splitting 2D Gaussians

TL;DR

This paper introduces a novel point cloud rendering method that predicts 2D Gaussians from point clouds, enabling effective rendering of sparse data with strong generalization across datasets and categories, achieving state-of-the-art results.

Contribution

The method predicts 2D Gaussians from point clouds using a patch-based architecture, improving rendering quality and generalization without relying on dense point clouds or additional refinements.

Findings

Achieves state-of-the-art rendering performance.

Effectively handles sparse point clouds.

Generalizes across different datasets and categories.

Abstract

Current learning-based methods predict NeRF or 3D Gaussians from point clouds to achieve photo-realistic rendering but still depend on categorical priors, dense point clouds, or additional refinements. Hence, we introduce a novel point cloud rendering method by predicting 2D Gaussians from point clouds. Our method incorporates two identical modules with an entire-patch architecture enabling the network to be generalized to multiple datasets. The module normalizes and initializes the Gaussians utilizing the point cloud information including normals, colors and distances. Then, splitting decoders are employed to refine the initial Gaussians by duplicating them and predicting more accurate results, making our methodology effectively accommodate sparse point clouds as well. Once trained, our approach exhibits direct generalization to point clouds across different categories. The predicted Gaussians are employed directly for rendering without additional refinement on the rendered images, retaining the benefits of 2D Gaussians. We conduct extensive experiments on various datasets, and the results demonstrate the superiority and generalization of our method, which achieves SOTA performance. The code is available at https://github.com/murcherful/GauPCRender}{https://github.com/murcherful/GauPCRender.