FHGS: Feature-Homogenized Gaussian Splatting

TL;DR

FHGS introduces a novel 3D feature fusion framework that enhances scene understanding by mapping 2D semantic features into 3D scenes while maintaining real-time rendering efficiency and ensuring viewpoint-independent feature consistency.

Contribution

The paper proposes FHGS, a new framework that effectively embeds large-scale pre-trained semantic features into 3D Gaussian splatting, balancing anisotropic rendering with isotropic semantic representations.

Findings

Achieves high-precision 3D semantic mapping

Maintains real-time rendering efficiency

Ensures viewpoint-independent feature consistency

Abstract

Scene understanding based on 3D Gaussian Splatting (3DGS) has recently achieved notable advances. Although 3DGS related methods have efficient rendering capabilities, they fail to address the inherent contradiction between the anisotropic color representation of gaussian primitives and the isotropic requirements of semantic features, leading to insufficient cross-view feature consistency. To overcome the limitation, we proposes $\textit{FHGS}$ (Feature-Homogenized Gaussian Splatting), a novel 3D feature fusion framework inspired by physical models, which can achieve high-precision mapping of arbitrary 2D features from pre-trained models to 3D scenes while preserving the real-time rendering efficiency of 3DGS. Specifically, our $\textit{FHGS}$ introduces the following innovations: Firstly, a universal feature fusion architecture is proposed, enabling robust embedding of large-scale pre-trained models' semantic features (e.g., SAM, CLIP) into sparse 3D structures. Secondly, a non-differentiable feature fusion mechanism is introduced, which enables semantic features to exhibit viewpoint independent isotropic distributions. This fundamentally balances the anisotropic rendering of gaussian primitives and the isotropic expression of features; Thirdly, a dual-driven optimization strategy inspired by electric potential fields is proposed, which combines external supervision from semantic feature fields with internal primitive clustering guidance. This mechanism enables synergistic optimization of global semantic alignment and local structural consistency. More interactive results can be accessed on: https://fhgs.cuastro.org/.