FlashVGGT: Efficient and Scalable Visual Geometry Transformers with Compressed Descriptor Attention

TL;DR

FlashVGGT introduces a descriptor-based attention mechanism for 3D reconstruction that maintains high accuracy while significantly improving scalability and inference efficiency over long image sequences.

Contribution

It proposes a novel compressed descriptor attention method that reduces computational complexity and enables efficient online inference in large-scale 3D reconstruction tasks.

Findings

Achieves comparable accuracy to VGGT in 3D reconstruction.

Reduces inference time to 9.3% of VGGT for 1,000 images.

Scales efficiently to sequences over 3,000 images.

Abstract

3D reconstruction from multi-view images is a core challenge in computer vision. Recently, feed-forward methods have emerged as efficient and robust alternatives to traditional per-scene optimization techniques. Among them, state-of-the-art models like the Visual Geometry Grounding Transformer (VGGT) leverage full self-attention over all image tokens to capture global relationships. However, this approach suffers from poor scalability due to the quadratic complexity of self-attention and the large number of tokens generated in long image sequences. In this work, we introduce FlashVGGT, an efficient alternative that addresses this bottleneck through a descriptor-based attention mechanism. Instead of applying dense global attention across all tokens, FlashVGGT compresses spatial information from each frame into a compact set of descriptor tokens. Global attention is then computed as cross-attention between the full set of image tokens and this smaller descriptor set, significantly reducing computational overhead. Moreover, the compactness of the descriptors enables online inference over long sequences via a chunk-recursive mechanism that reuses cached descriptors from previous chunks. Experimental results show that FlashVGGT achieves reconstruction accuracy competitive with VGGT while reducing inference time to just 9.3% of VGGT for 1,000 images, and scaling efficiently to sequences exceeding 3,000 images. Our project page is available at https://wzpscott.github.io/flashvggt_page/.