Deep Patch Visual Odometry

TL;DR

Deep Patch Visual Odometry (DPVO) introduces a recurrent patch-based matching system that achieves superior accuracy and efficiency in monocular VO, outperforming dense flow-based methods with less computational cost.

Contribution

DPVO demonstrates that sparse patch-based matching can surpass dense flow approaches in accuracy and efficiency for visual odometry, challenging previous assumptions.

Findings

DPVO outperforms prior VO systems on benchmarks.

DPVO uses a third of the memory of dense flow methods.

DPVO runs three times faster than previous state-of-the-art.

Abstract

We propose Deep Patch Visual Odometry (DPVO), a new deep learning system for monocular Visual Odometry (VO). DPVO uses a novel recurrent network architecture designed for tracking image patches across time. Recent approaches to VO have significantly improved the state-of-the-art accuracy by using deep networks to predict dense flow between video frames. However, using dense flow incurs a large computational cost, making these previous methods impractical for many use cases. Despite this, it has been assumed that dense flow is important as it provides additional redundancy against incorrect matches. DPVO disproves this assumption, showing that it is possible to get the best accuracy and efficiency by exploiting the advantages of sparse patch-based matching over dense flow. DPVO introduces a novel recurrent update operator for patch based correspondence coupled with differentiable bundle adjustment. On Standard benchmarks, DPVO outperforms all prior work, including the learning-based state-of-the-art VO-system (DROID) using a third of the memory while running 3x faster on average. Code is available at https://github.com/princeton-vl/DPVO