VersaQ-3D: A Reconfigurable Accelerator Enabling Feed-Forward and Generalizable 3D Reconstruction via Versatile Quantization

TL;DR

VersaQ-3D introduces a reconfigurable accelerator and a novel quantization method for VGGT, enabling efficient, accurate, and low-power 3D reconstruction on edge devices without scene-specific calibration.

Contribution

It presents the first calibration-free, scene-agnostic 4-bit quantization for VGGT and a reconfigurable hardware accelerator supporting multiple precisions, improving efficiency and deployment feasibility.

Findings

Preserves 98-99% accuracy at W4A8 quantization.

Outperforms prior methods by 1.61x-2.39x at W4A4.

Achieves 5.2x-10.8x speedup over edge GPUs.

Abstract

The Visual Geometry Grounded Transformer (VGGT) enables strong feed-forward 3D reconstruction without per-scene optimization. However, its billion-parameter scale creates high memory and compute demands, hindering on-device deployment. Existing LLM quantization methods fail on VGGT due to saturated activation channels and diverse 3D semantics, which cause unreliable calibration. Furthermore, VGGT presents hardware challenges regarding precision-sensitive nonlinear operators and memory-intensive global attention. To address this, we propose VersaQ-3D, an algorithm-architecture co-design framework. Algorithmically, we introduce the first calibration-free, scene-agnostic quantization for VGGT down to 4-bit, leveraging orthogonal transforms to decorrelate features and suppress outliers. Architecturally, we design a reconfigurable accelerator supporting BF16, INT8, and INT4. A unified systolic datapath handles both linear and nonlinear operators, reducing latency by 60%, while two-stage recomputation-based tiling alleviates memory pressure for long-sequence attention. Evaluations show VersaQ-3D preserves 98-99% accuracy at W4A8. At W4A4, it outperforms prior methods by 1.61x-2.39x across diverse scenes. The accelerator delivers 5.2x-10.8x speedup over edge GPUs with low power, enabling efficient instant 3D reconstruction.