FLEG: Feed-Forward Language Embedded Gaussian Splatting from Any Views via Compact Semantic Representation

TL;DR

FLEG introduces a novel feed-forward approach for reconstructing language-embedded 3D Gaussians from arbitrary views, reducing storage costs and improving semantic fidelity without fixed input view constraints.

Contribution

The paper proposes a geometry-semantic dual-branch framework and a sparse language embedding strategy, enabling flexible multi-view input and efficient semantic representation.

Findings

FLEG outperforms existing methods in reconstruction quality.

Uses only 5% of the language embeddings compared to dense schemes.

Maintains semantic fidelity with significantly reduced storage costs.

Abstract

We present FLEG, a feed-forward network that reconstructs language-embedded 3D Gaussians from arbitrary views. Previous feed-forward language-embedded Gaussian reconstruction methods are restricted to a fixed number of input views and typically attach a language-aligned semantic embedding to each Gaussian, resulting in impractical input settings and semantic redundancy. In contrast, we introduce a geometry-semantic dual-branch distillation framework that enables flexible input from arbitrary multi-view images without camera parameters. We also propose a novel-view-based distillation strategy during training that mitigates overfitting to input views. In addition, we observe that semantic representations are significantly sparser than geometric ones, and per-Gaussian language embedding is unnecessary. To exploit this sparsity, we design a decoupled language embedding strategy that represents language information with a sparse set of semantic Gaussians, rather than attaching embeddings to every Gaussian. Compared with dense pixel-aligned per-Gaussian embedding schemes, our method uses only 5\% of the language embeddings while maintaining comparable semantic fidelity, effectively reducing storage costs. Extensive experiments demonstrate that FLEG outperforms state-of-the-art feed-forward reconstruction and language-embedded Gaussian methods in both reconstruction quality and language-aligned semantic representation. Project page: https://fangzhou2000.github.io/projects/fleg.