SSR: A Training-Free Approach for Streaming 3D Reconstruction

TL;DR

This paper introduces SSR, a training-free method that enforces geometric consistency in streaming 3D reconstruction by regularizing state trajectories on the Grassmannian manifold, reducing drift and improving accuracy.

Contribution

The paper proposes SSR, a novel inference-time regularization technique that maintains state coherence on the Grassmannian manifold without additional training.

Findings

SSR reduces geometric drift in long-sequence 3D reconstructions.

SSR improves reconstruction quality across multiple benchmarks.

SSR operates with minimal computational overhead.

Abstract

Streaming 3D reconstruction demands long-horizon state updates under strict latency constraints, yet stateful recurrent models often suffer from geometric drift as errors accumulate over time. We revisit this problem from a Grassmannian manifold perspective: the latent persistent state can be viewed as a subspace representation, i.e., a point evolving on a Grassmannian manifold, where temporal coherence implies the state trajectory should remain on (or near) this manifold.Based on this view, we propose Self-expressive Sequence Regularization (SSR), a plug-and-play, training-free operator that enforces Grassmannian sequence regularity during inference.Given a window of historical states, SSR computes an analytical affinity matrix via the self-expressive property and uses it to regularize the current update, effectively pulling noisy predictions back toward the manifold-consistent trajectory with minimal overhead. Experiments on long-sequence benchmarks demonstrate that SSR consistently reduces drift and improves reconstruction quality across multiple streaming 3D reconstruction tasks.