CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes

TL;DR

CD-NGP introduces a scalable continual learning framework for dynamic scene view synthesis, reducing memory and improving quality through novel encoding techniques and a new long-duration video dataset.

Contribution

It proposes a continual learning approach with spatial-temporal hash encodings for dynamic scenes, and introduces a new long-duration video dataset for benchmarking.

Findings

Reduces training memory to under 14GB.

Achieves high-quality rendering with only 0.4MB/frame bandwidth.

Outperforms existing methods in scalability and reconstruction quality.

Abstract

Novel view synthesis (NVS) in dynamic scenes faces persistent challenges in memory consumption, model complexity, training efficiency, and rendering quality. Offline methods offer high fidelity but suffer from high memory usage and limited scalability, while online approaches often trade quality for speed and compactness. We propose Continual Dynamic Neural Graphics Primitives (CD-NGP), a continual learning framework that reduces memory overhead and enhances scalability through parameter reuse. To avoid feature interference in dynamic scenes and improve rendering quality, our method combines spatial and temporal hash encodings, which compactly represent scene structures and motion patterns. We also introduce a new dataset comprising multi-view, long-duration ($>1200$ frames) videos with both rigid and non-rigid motion, which is not found in existing benchmarks. CD-NGP is evaluated on public datasets and our long video dataset, demonstrating superior scalability and reconstruction quality. It significantly reduces training memory usage to <14GB and requires only 0.4MB/frame in streaming bandwidth on DyNeRF -- substantially lower than most online baselines.