3D Gaussian Point Encoders

TL;DR

This paper introduces 3D Gaussian Point Encoders, explicit geometric representations for 3D recognition that are faster, more efficient, and competitive with PointNets, leveraging optimization techniques and geometric heuristics.

Contribution

The paper presents a novel 3D Gaussian Point Encoder that is faster, more parameter-efficient, and integrates optimization and filtering techniques for improved 3D recognition.

Findings

Faster encoding with 2.7x speedup over PointNet.

Uses 46% less memory and 88% fewer FLOPs.

Achieves high framerates on CPU-only devices.

Abstract

In this work, we introduce the 3D Gaussian Point Encoder, an explicit per-point embedding built on mixtures of learned 3D Gaussians. This explicit geometric representation for 3D recognition tasks is a departure from widely used implicit representations such as PointNet. However, it is difficult to learn 3D Gaussian encoders in end-to-end fashion with standard optimizers. We develop optimization techniques based on natural gradients and distillation from PointNets to find a Gaussian Basis that can reconstruct PointNet activations. The resulting 3D Gaussian Point Encoders are faster and more parameter efficient than traditional PointNets. As in the 3D reconstruction literature where there has been considerable interest in the move from implicit (e.g., NeRF) to explicit (e.g., Gaussian Splatting) representations, we can take advantage of computational geometry heuristics to accelerate 3D Gaussian Point Encoders further. We extend filtering techniques from 3D Gaussian Splatting to construct encoders that run 2.7 times faster as a comparable accuracy PointNet while using 46% less memory and 88% fewer FLOPs. Furthermore, we demonstrate the effectiveness of 3D Gaussian Point Encoders as a component in Mamba3D, running 1.27 times faster and achieving a reduction in memory and FLOPs by 42% and 54% respectively. 3D Gaussian Point Encoders are lightweight enough to achieve high framerates on CPU-only devices.