HyVGGT-VO: Tightly Coupled Hybrid Dense Visual Odometry with Feed-Forward Models

TL;DR

HyVGGT-VO is a novel dense visual odometry framework that combines traditional sparse methods with feed-forward models for real-time dense mapping, achieving significant speed and accuracy improvements.

Contribution

This work introduces the first tight coupling of traditional VO with VGGT feed-forward models, including an adaptive hybrid tracker and hierarchical optimization for global scale consistency.

Findings

Achieves 5x speedup over existing VGGT methods.

Reduces trajectory error by 85% on EuRoC dataset.

Reduces trajectory error by 12% on KITTI benchmark.

Abstract

Dense visual odometry (VO), which provides pose estimation and dense 3D reconstruction, serves as the cornerstone for applications ranging from robotics to augmented reality. Recently, feed-forward models have demonstrated remarkable capabilities in dense mapping. However, when these models are used in dense visual SLAM systems, their heavy computational burden restricts them to yielding sparse pose outputs at keyframes while still failing to achieve real-time pose estimation. In contrast, traditional sparse methods provide high computational efficiency and high-frequency pose outputs, but lack the capability for dense reconstruction. To address these limitations, we propose HyVGGT-VO, a novel framework that combines the computational efficiency of sparse VO with the dense reconstruction capabilities of feed-forward models. To the best of our knowledge, this is the first work to tightly couple a traditional VO framework with VGGT, a state-of-the-art feed-forward model. Specifically, we design an adaptive hybrid tracking frontend that dynamically switches between traditional optical flow and the VGGT tracking head to ensure robustness. Furthermore, we introduce a hierarchical optimization framework that jointly refines VO poses and the scale of VGGT predictions to ensure global scale consistency. Our approach achieves an approximately 5x processing speedup compared to existing VGGT-based methods, while reducing the average trajectory error by 85% on the indoor EuRoC dataset and 12% on the outdoor KITTI benchmark. Our code will be publicly available upon acceptance. Project page: https://geneta2580.github.io/HyVGGT-VO.io.