A Bag of Tricks for Efficient Implicit Neural Point Clouds

TL;DR

This paper introduces a set of optimizations for Implicit Neural Point Clouds that significantly enhance training and rendering speed while maintaining high visual quality, making INPC more practical for real-world applications.

Contribution

The paper presents novel optimization techniques, including improved rasterization, sampling, pre-training, and Gaussian modeling, to accelerate INPC training and inference without quality loss.

Findings

Up to 25% faster training

2x faster rendering speed

20% reduction in VRAM usage

Abstract

Implicit Neural Point Cloud (INPC) is a recent hybrid representation that combines the expressiveness of neural fields with the efficiency of point-based rendering, achieving state-of-the-art image quality in novel view synthesis. However, as with other high-quality approaches that query neural networks during rendering, the practical usability of INPC is limited by comparatively slow rendering. In this work, we present a collection of optimizations that significantly improve both the training and inference performance of INPC without sacrificing visual fidelity. The most significant modifications are an improved rasterizer implementation, more effective sampling techniques, and the incorporation of pre-training for the convolutional neural network used for hole-filling. Furthermore, we demonstrate that points can be modeled as small Gaussians during inference to further improve quality in extrapolated, e.g., close-up views of the scene. We design our implementations to be broadly applicable beyond INPC and systematically evaluate each modification in a series of experiments. Our optimized INPC pipeline achieves up to 25% faster training, 2x faster rendering, and 20% reduced VRAM usage paired with slight image quality improvements.