BEA-GS: BEyond RAdiance Supervision in 3DGS for Precise Object Extraction

TL;DR

This paper introduces BEA-GS, a novel Gaussian Splatting method that optimizes 3D geometry for precise object boundary extraction, outperforming existing techniques across multiple datasets.

Contribution

It proposes two new loss functions that improve geometry optimization for object boundaries in Gaussian Splatting, enabling near-perfect object extraction.

Findings

Achieves the best boundary segmentation results among 12 state-of-the-art methods.

Demonstrates superior performance across 4 datasets using six metrics.

Abstract

Most Gaussian Splatting techniques that provide a 3D semantic representation of the scene do not optimize the underlying 3D geometry, making object-level editing or asset extraction challenging. Recent methods, such as COBGS, Trace3D, ObjectGS, acknowledge this limitation and propose approaches that modify the scene's geometry to represent the underlying semantics. We advance this concept further by proposing a novel solution that provides near perfect boundaries in object extraction. We do so by introducing two new losses in the optimization that take care of: 1) a loss that modifies the geometry of visible Gaussians to respect semantic boundaries, and 2) a loss that adjusts the geometry of non-visible Gaussians that appear once the object is extracted. Our first loss propagates gradients directly through the rasterization, allowing for seamless integration within the optimization of the Gaussian parameters. The second loss also propagates gradients to Gaussian parameters but does so without passing through the rasterization, enabling modification of the scene's geometry even when little transmittance reaches a Gaussian (partial or non-visible). Exhaustive comparisons with 12 state of the art methods across 4 datasets, using six metrics, demonstrate that our approach produces overall the best boundary segmentation to date.