VGF-Net: Visual-Geometric Fusion Learning for Simultaneous Drone Navigation and Height Mapping

TL;DR

VGF-Net is a deep learning framework that fuses visual and geometric data to improve drone navigation and height mapping in complex environments, demonstrating high accuracy and robustness.

Contribution

The paper introduces VGF-Net, a novel deep network that adaptively fuses visual and geometric information for improved 3D mapping and drone navigation.

Findings

Achieves high accuracy in complex indoor and outdoor scenes.

Demonstrates robustness in dynamic environments.

Provides an end-to-end system for real-time navigation and mapping.

Abstract

The drone navigation requires the comprehensive understanding of both visual and geometric information in the 3D world. In this paper, we present a Visual-Geometric Fusion Network(VGF-Net), a deep network for the fusion analysis of visual/geometric data and the construction of 2.5D height maps for simultaneous drone navigation in novel environments. Given an initial rough height map and a sequence of RGB images, our VGF-Net extracts the visual information of the scene, along with a sparse set of 3D keypoints that capture the geometric relationship between objects in the scene. Driven by the data, VGF-Net adaptively fuses visual and geometric information, forming a unified Visual-Geometric Representation. This representation is fed to a new Directional Attention Model(DAM), which helps enhance the visual-geometric object relationship and propagates the informative data to dynamically refine the height map and the corresponding keypoints. An entire end-to-end information fusion and mapping system is formed, demonstrating remarkable robustness and high accuracy on the autonomous drone navigation across complex indoor and large-scale outdoor scenes. The dataset can be found in http://vcc.szu.edu.cn/research/2021/VGFNet.