RobustGS: Unified Boosting of Feedforward 3D Gaussian Splatting under Low-Quality Conditions

TL;DR

RobustGS introduces a plug-and-play module that significantly enhances the robustness of feedforward 3D Gaussian Splatting under challenging imaging conditions, leading to improved 3D reconstruction quality.

Contribution

The paper proposes RobustGS, a novel multi-view feature enhancement module with a Generalized Degradation Learner and semantic-aware state-space model, improving robustness of 3DGS under adverse conditions.

Findings

Achieves state-of-the-art reconstruction quality under various degradations.

Seamlessly integrates into existing 3DGS pipelines.

Enhances feature representations for better cross-view correspondence.

Abstract

Feedforward 3D Gaussian Splatting (3DGS) overcomes the limitations of optimization-based 3DGS by enabling fast and high-quality reconstruction without the need for per-scene optimization. However, existing feedforward approaches typically assume that input multi-view images are clean and high-quality. In real-world scenarios, images are often captured under challenging conditions such as noise, low light, or rain, resulting in inaccurate geometry and degraded 3D reconstruction. To address these challenges, we propose a general and efficient multi-view feature enhancement module, RobustGS, which substantially improves the robustness of feedforward 3DGS methods under various adverse imaging conditions, enabling high-quality 3D reconstruction. The RobustGS module can be seamlessly integrated into existing pretrained pipelines in a plug-and-play manner to enhance reconstruction robustness. Specifically, we introduce a novel component, Generalized Degradation Learner, designed to extract generic representations and distributions of multiple degradations from multi-view inputs, thereby enhancing degradation-awareness and improving the overall quality of 3D reconstruction. In addition, we propose a novel semantic-aware state-space model. It first leverages the extracted degradation representations to enhance corrupted inputs in the feature space. Then, it employs a semantic-aware strategy to aggregate semantically similar information across different views, enabling the extraction of fine-grained cross-view correspondences and further improving the quality of 3D representations. Extensive experiments demonstrate that our approach, when integrated into existing methods in a plug-and-play manner, consistently achieves state-of-the-art reconstruction quality across various types of degradations.