MUT3R: Motion-aware Updating Transformer for Dynamic 3D Reconstruction

TL;DR

MUT3R is a training-free, motion-aware framework that improves dynamic 3D reconstruction by leveraging attention cues in pretrained transformers to suppress motion artifacts without retraining.

Contribution

The paper introduces MUT3R, a novel, training-free method that uses attention-derived motion cues to enhance dynamic 3D reconstruction stability and accuracy.

Findings

Improves temporal consistency in dynamic 3D reconstruction.

Enhances robustness to camera pose variations.

Does not require retraining or fine-tuning the transformer.

Abstract

Recent stateful recurrent neural networks have achieved remarkable progress on static 3D reconstruction but remain vulnerable to motion-induced artifacts, where non-rigid regions corrupt attention propagation between the spatial memory and image feature. By analyzing the internal behaviors of the state and image token updating mechanism, we find that aggregating self-attention maps across layers reveals a consistent pattern: dynamic regions are naturally down-weighted, exposing an implicit motion cue that the pretrained transformer already encodes but never explicitly uses. Motivated by this observation, we introduce MUT3R, a training-free framework that applies the attention-derived motion cue to suppress dynamic content in the early layers of the transformer during inference. Our attention-level gating module suppresses the influence of dynamic regions before their artifacts propagate through the feature hierarchy. Notably, we do not retrain or fine-tune the model; we let the pretrained transformer diagnose its own motion cues and correct itself. This early regulation stabilizes geometric reasoning in streaming scenarios and leads to improvements in temporal consistency and camera pose robustness across multiple dynamic benchmarks, offering a simple and training-free pathway toward motion-aware streaming reconstruction.