LongStream: Long-Sequence Streaming Autoregressive Visual Geometry

TL;DR

LongStream introduces a novel online method for long-sequence 3D scene reconstruction that effectively addresses scale drift and attention issues, enabling stable, kilometer-scale reconstructions at real-time speeds.

Contribution

The paper presents a gauge-decoupled, scale-disentangled autoregressive model with attention bias mitigation for ultra-long streaming 3D reconstruction.

Findings

Achieves state-of-the-art long-sequence reconstruction accuracy.

Maintains stable, metric-scale reconstructions over kilometer sequences.

Operates at 18 FPS in real-time.

Abstract

Long-sequence streaming 3D reconstruction remains a significant open challenge. Existing autoregressive models often fail when processing long sequences because they anchor poses to the first frame, leading to attention decay, scale drift, and extrapolation errors. We introduce LongStream, a novel gauge-decoupled streaming visual geometry model for metric-scale scene reconstruction across thousands of frames under a strictly online, future-invisible setting. Our approach is threefold. First, we discard the first-frame anchor and predict keyframe-relative poses. This reformulates long-range extrapolation into a constant-difficulty local task. Second, we introduce orthogonal scale learning. This method fully disentangles geometry from scale estimation to suppress drift. Finally, we identify attention bias issues in Transformers, including attention-sink reliance and long-term KV-cache saturation. We propose cache-consistent training combined with periodic cache refresh. This approach suppresses attention biases and contamination over ultra-long sequences and reduces the gap between training and inference. Experiments show that LongStream achieves state-of-the-art performance, enabling stable, metric-scale reconstruction over kilometer-scale sequences at 18 FPS. Project Page: https://3dagentworld.github.io/longstream/