SuperGS: Super-Resolution 3D Gaussian Splatting Enhanced by Variational Residual Features and Uncertainty-Augmented Learning

TL;DR

SuperGS enhances 3D Gaussian Splatting for high-resolution view synthesis by employing a two-stage training framework, variational residual features, and multi-view learning, achieving superior results with low-resolution inputs.

Contribution

It introduces a novel two-stage coarse-to-fine training framework with variational residual features and multi-view learning for high-resolution 3D scene synthesis.

Findings

Outperforms state-of-the-art HRNVS methods on real-world datasets.

Effectively utilizes low-resolution inputs for high-quality 3D scene rendering.

Demonstrates robustness through extensive experiments.

Abstract

Recently, 3D Gaussian Splatting (3DGS) has exceled in novel view synthesis (NVS) with its real-time rendering capabilities and superior quality. However, it faces challenges for high-resolution novel view synthesis (HRNVS) due to the coarse nature of primitives derived from low-resolution input views. To address this issue, we propose Super-Resolution 3DGS (SuperGS), which is an expansion of 3DGS designed with a two-stage coarse-to-fine training framework. In this framework, we use a latent feature field to represent the low-resolution scene, serving as both the initialization and foundational information for super-resolution optimization. Additionally, we introduce variational residual features to enhance high-resolution details, using their variance as uncertainty estimates to guide the densification process and loss computation. Furthermore, the introduction of a multi-view joint learning approach helps mitigate ambiguities caused by multi-view inconsistencies in the pseudo labels. Extensive experiments demonstrate that SuperGS surpasses state-of-the-art HRNVS methods on both real-world and synthetic datasets using only low-resolution inputs. Code is available at https://github.com/SYXieee/SuperGS.