Keyframe-Based Feed-Forward Visual Odometry

TL;DR

This paper introduces a reinforcement learning-based keyframe selection method for feed-forward visual odometry, improving efficiency and accuracy by reducing redundancy and leveraging adaptive keyframe policies.

Contribution

It presents a novel keyframe-based approach that integrates reinforcement learning to adaptively select keyframes, enhancing existing foundation model-based visual odometry methods.

Findings

Significant performance improvements over state-of-the-art methods

Efficient reduction of computational redundancy

Robustness across multiple real-world datasets

Abstract

The emergence of visual foundation models has revolutionized visual odometry~(VO) and SLAM, enabling pose estimation and dense reconstruction within a single feed-forward network. However, unlike traditional pipelines that leverage keyframe methods to enhance efficiency and accuracy, current foundation model based methods, such as VGGT-Long, typically process raw image sequences indiscriminately. This leads to computational redundancy and degraded performance caused by low inter-frame parallax, which provides limited contextual stereo information. Integrating traditional geometric heuristics into these methods is non-trivial, as their performance depends on high-dimensional latent representations rather than explicit geometric metrics. To bridge this gap, we propose a novel keyframe-based feed-forward VO. Instead of relying on hand-crafted rules, our approach employs reinforcement learning to derive an adaptive keyframe policy in a data-driven manner, aligning selection with the intrinsic characteristics of the underlying foundation model. We train our agent on TartanAir dataset and conduct extensive evaluations across several real-world datasets. Experimental results demonstrate that the proposed method achieves consistent and substantial improvements over state-of-the-art feed-forward VO methods.