CF3: Compact and Fast 3D Feature Fields

TL;DR

CF3 introduces a top-down method for creating compact, efficient 3D Gaussian feature fields by fusing multi-view 2D features and adaptively pruning Gaussians, significantly reducing complexity while maintaining detail.

Contribution

The paper presents a novel top-down pipeline for 3D Gaussian feature fields, enabling faster, more compact representations through multi-view fusion and adaptive sparsification.

Findings

Achieves comparable 3D feature quality with only 5% of Gaussians.

Reduces computational costs compared to bottom-up approaches.

Maintains geometric detail despite significant sparsification.

Abstract

3D Gaussian Splatting (3DGS) has begun incorporating rich information from 2D foundation models. However, most approaches rely on a bottom-up optimization process that treats raw 2D features as ground truth, incurring increased computational costs. We propose a top-down pipeline for constructing compact and fast 3D Gaussian feature fields, namely, CF3. We first perform a fast weighted fusion of multi-view 2D features with pre-trained Gaussians. This approach enables training a per-Gaussian autoencoder directly on the lifted features, instead of training autoencoders in the 2D domain. As a result, the autoencoder better aligns with the feature distribution. More importantly, we introduce an adaptive sparsification method that optimizes the Gaussian attributes of the feature field while pruning and merging the redundant Gaussians, constructing an efficient representation with preserved geometric details. Our approach achieves a competitive 3D feature field using as little as 5% of the Gaussians compared to Feature-3DGS.