Adaptive Anchor Policies for Efficient 4D Gaussian Streaming

TL;DR

This paper introduces EGS, a reinforcement learning-based anchor selection method for 4D Gaussian streaming, improving efficiency and quality in dynamic scene reconstruction over fixed sampling methods.

Contribution

EGS replaces fixed anchor sampling with a learned policy, enabling adaptive, budget-aware selection that enhances reconstruction quality and runtime efficiency.

Findings

EGS improves PSNR by 0.52-0.61 dB at 256 anchors compared to fixed sampling.

EGS runs 1.29-1.35 times faster than baseline at large anchor counts.

EGS maintains competitive quality at lower anchor budgets in high-quality refinement.

Abstract

Dynamic scene reconstruction with Gaussian Splatting has enabled efficient streaming for real-time rendering and free-viewpoint video. However, most pipelines rely on fixed anchor selection such as Farthest Point Sampling (FPS), typically using 8,192 anchors regardless of scene complexity, which over-allocates computation under strict budgets. We propose Efficient Gaussian Streaming (EGS), a plug-in, budget-aware anchor sampler that replaces FPS with a reinforcement-learned policy while keeping the Gaussian streaming reconstruction backbone unchanged. The policy jointly selects an anchor budget and a subset of informative anchors under discrete constraints, balancing reconstruction quality and runtime using spatial features of the Gaussian representation. We evaluate EGS in two settings: fast rendering, which prioritizes runtime efficiency, and high-quality refinement, which enables additional optimization. Experiments on dynamic multi-view datasets show consistent improvements in the quality--efficiency trade-off over FPS sampling. On unseen data, in fast rendering at 256 anchors ($32\times$ fewer than 8,192), EGS improves PSNR by $+0.52$--$0.61$\,dB while running $1.29$--$1.35\times$ faster than IGS@8192 (N3DV and MeetingRoom). In high-quality refinement, EGS remains competitive with the full-anchor baseline at substantially lower anchor budgets. \emph{Code and pretrained checkpoints will be released upon acceptance.} \keywords{4D Gaussian Splatting \and 4D Gaussian Streaming \and Reinforcement Learning}