PFGS: High Fidelity Point Cloud Rendering via Feature Splatting

TL;DR

This paper introduces PFGS, a novel point cloud rendering framework that combines Gaussian Splatting and neural feature rasterization to produce high-fidelity images from sparse data, overcoming artifacts and missing details.

Contribution

It proposes a new cascaded pipeline with a regressor, multiscale feature extraction, and progressive decoding, achieving superior rendering quality from sparse point clouds.

Findings

Outperforms existing methods in rendering quality on benchmarks.

Effectively reduces artifacts and preserves details in sparse point cloud rendering.

Demonstrates the importance of multiscale and progressive decoding components.

Abstract

Rendering high-fidelity images from sparse point clouds is still challenging. Existing learning-based approaches suffer from either hole artifacts, missing details, or expensive computations. In this paper, we propose a novel framework to render high-quality images from sparse points. This method first attempts to bridge the 3D Gaussian Splatting and point cloud rendering, which includes several cascaded modules. We first use a regressor to estimate Gaussian properties in a point-wise manner, the estimated properties are used to rasterize neural feature descriptors into 2D planes which are extracted from a multiscale extractor. The projected feature volume is gradually decoded toward the final prediction via a multiscale and progressive decoder. The whole pipeline experiences a two-stage training and is driven by our well-designed progressive and multiscale reconstruction loss. Experiments on different benchmarks show the superiority of our method in terms of rendering qualities and the necessities of our main components.