Extreme Views: 3DGS Filter for Novel View Synthesis from Out-of-Distribution Camera Poses

TL;DR

This paper introduces a real-time filtering technique for 3D Gaussian Splatting models that enhances visual quality and stability when viewing from out-of-distribution camera angles, addressing artifacts caused by uncertain predictions.

Contribution

It proposes a novel, real-time, render-aware filtering method based on sensitivity scores to improve out-of-distribution view synthesis in 3DGS models, without retraining.

Findings

Significantly improves visual quality and realism in out-of-distribution views.

Seamlessly integrates into existing 3DGS pipelines in real-time.

Outperforms existing NeRF-based approaches like BayesRays.

Abstract

When viewing a 3D Gaussian Splatting (3DGS) model from camera positions significantly outside the training data distribution, substantial visual noise commonly occurs. These artifacts result from the lack of training data in these extrapolated regions, leading to uncertain density, color, and geometry predictions from the model. To address this issue, we propose a novel real-time render-aware filtering method. Our approach leverages sensitivity scores derived from intermediate gradients, explicitly targeting instabilities caused by anisotropic orientations rather than isotropic variance. This filtering method directly addresses the core issue of generative uncertainty, allowing 3D reconstruction systems to maintain high visual fidelity even when users freely navigate outside the original training viewpoints. Experimental evaluation demonstrates that our method substantially improves visual quality, realism, and consistency compared to existing Neural Radiance Field (NeRF)-based approaches such as BayesRays. Critically, our filter seamlessly integrates into existing 3DGS rendering pipelines in real-time, unlike methods that require extensive post-hoc retraining or fine-tuning. Code and results at https://damian-bowness.github.io/EV3DGS