GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments

TL;DR

GaussianUpdate is a new method for real-time view synthesis that adaptively updates 3D Gaussian models to reflect scene changes over time without extensive retraining.

Contribution

It introduces a multi-stage update strategy and a visibility-aware continual learning approach with generative replay for dynamic scene adaptation.

Findings

Achieves superior real-time rendering quality.

Effectively models different types of scene changes.

Does not require storing images for updates.

Abstract

Novel view synthesis with neural models has advanced rapidly in recent years, yet adapting these models to scene changes remains an open problem. Existing methods are either labor-intensive, requiring extensive model retraining, or fail to capture detailed types of changes over time. In this paper, we present GaussianUpdate, a novel approach that combines 3D Gaussian representation with continual learning to address these challenges. Our method effectively updates the Gaussian radiance fields with current data while preserving information from past scenes. Unlike existing methods, GaussianUpdate explicitly models different types of changes through a novel multi-stage update strategy. Additionally, we introduce a visibility-aware continual learning approach with generative replay, enabling self-aware updating without the need to store images. The experiments on the benchmark dataset demonstrate our method achieves superior and real-time rendering with the capability of visualizing changes over different times