Semantic Nearest Neighbor Fields Monocular Edge Visual-Odometry

TL;DR

This paper introduces a monocular visual odometry system that leverages semantic edges and nearest neighbor fields to improve robustness and accuracy in outdoor environments, enabling large-scale semantic mapping.

Contribution

It proposes a novel semantic nearest neighbor field for robust edge association, enhancing monocular VO with semantic, geometric, and photometric consistency.

Findings

Outperforms state-of-the-art monocular VO systems

Enables large-scale semantic map reconstruction

Improves robustness in challenging outdoor environments

Abstract

Recent advances in deep learning for edge detection and segmentation opens up a new path for semantic-edge-based ego-motion estimation. In this work, we propose a robust monocular visual odometry (VO) framework using category-aware semantic edges. It can reconstruct large-scale semantic maps in challenging outdoor environments. The core of our approach is a semantic nearest neighbor field that facilitates a robust data association of edges across frames using semantics. This significantly enlarges the convergence radius during tracking phases. The proposed edge registration method can be easily integrated into direct VO frameworks to estimate photometrically, geometrically, and semantically consistent camera motions. Different types of edges are evaluated and extensive experiments demonstrate that our proposed system outperforms state-of-art indirect, direct, and semantic monocular VO systems.