ViBA: Implicit Bundle Adjustment with Geometric and Temporal Consistency for Robust Visual Matching

TL;DR

ViBA is a novel learning framework that combines geometric optimization with feature learning for robust, real-time visual matching and localization in unconstrained video streams.

Contribution

It introduces an implicit bundle adjustment approach integrated into a visual odometry pipeline, enabling continuous online learning with geometric and temporal consistency.

Findings

Reduces translation error by 12-18% compared to state-of-the-art methods.

Maintains over 90% localization accuracy on unseen sequences.

Operates at real-time speeds of 36-91 FPS.

Abstract

Most existing image keypoint detection and description methods rely on datasets with accurate pose and depth annotations, limiting scalability and generalization, and often degrading navigation and localization performance. We propose ViBA, a sustainable learning framework that integrates geometric optimization with feature learning for continuous online training on unconstrained video streams. Embedded in a standard visual odometry pipeline, it consists of an implicitly differentiable geometric residual framework: (i) an initial tracking network for inter-frame correspondences, (ii) depth-based outlier filtering, and (iii) differentiable global bundle adjustment that jointly refines camera poses and feature positions by minimizing reprojection errors. By combining geometric consistency from BA with long-term temporal consistency across frames, ViBA enforces stable and accurate feature representations. We evaluate ViBA on EuRoC and UMA datasets. Compared with state-of-the-art methods such as SuperPoint+SuperGlue, ALIKED, and LightGlue, ViBA reduces mean absolute translation error (ATE) by 12-18% and absolute rotation error (ARE) by 5-10% across sequences, while maintaining real-time inference speeds (FPS 36-91). When evaluated on unseen sequences, it retains over 90% localization accuracy, demonstrating robust generalization. These results show that ViBA supports continuous online learning with geometric and temporal consistency, consistently improving navigation and localization in real-world scenarios.